While processing data quickly is crucial, making it accessible and meaningful to different applications and services within the organization poses a challenge. By combining the scalability and speed of a streaming database with the unified representation provided by a semantic layer, organizations can fully leverage their real-time streaming data.
In this blog, we will explore the integration of RisingWave, a database optimized for streaming data, and Cube, a universal semantic layer, and the tangible benefits it brings to data processing, consumption, and decision-making.
What is a semantic layer?
A universal semantic layer is an independent yet interoperable part of the modern data stack that sits between your data sources and data consumers. The universal semantic layer allows every data endpoint, whether it is BI tools, embedded analytics, or AI agents and chatbots, to work with the same semantics and underlying data, leading to consistent and trusted insights.It empowers users to interact with and analyze data without requiring in-depth knowledge of its underlying intricacies or structure.
The main goals of a semantic layer are to:
- Unify fragmented business definitions. Consolidate data modeling workflows and create a single source of truth with consistent metrics, defined once.
- Centralize and enforce fine-grained governance and security policies. Grant row and column level permissions and mask sensitive data upstream.
- Achieve faster, more cost-efficient results. Optimize query performance with a powerful caching layer and advanced pre-aggregation capabilities.
- Integrate with any endpoint. Use AI, GraphQL, REST, and SQL APIs to deliver trusted data to front-end applications.
Introducing Cube
Cube is a universal semantic layer that helps companies manage and optimize their data and analytics workflows. Any data source can be optimized for performance, accuracy, and consistency before being fed into any data application: internal, external, human, or bot-facing. Their universal semantic layer offers a wide range of capabilities including data modeling, data access control, caching, and instant APIs.
Introducing RisingWave
RisingWave is a database that empowers users to query and process streaming data using SQL in the style of Postgres. It utilizes incremental computation algorithms to minimize latency and ensures up-to-date results through real-time materialized views.
Why Using Cube with RisingWave?
RisingWave provides up-to-the-minute results through SQL, but some users want to access that real-time data through other interfaces like GraphQL APIs. This can make building event-driven applications much easier. Cube is an excellent solution for this.
GraphQL has some neat advantages over traditional REST APIs. It's more efficient since you only fetch the data you need, reducing network traffic and database load. It's also more flexible, allowing clients to describe exactly what data they want in a single request.
In RisingWave, results of materialized views typically represent the latest values of key metrics. Being able to access these key metrics is critical for real-time applications and analytics.
By using Cube as a GraphQL gateway for RisingWave, you can turn those fresh materialized view results into a productized API without writing any code yourself. Just configure your data model, and Cube automatically generates a blazing-fast GraphQL API on top of RisingWave's data store.
Demo: Integrate RisingWave Cloud with Cube Cloud
Both RisingWave and Cube offer cloud-hosted versions now. Let’s see how we can use them together to turn real-time insights into APIs.
Set up RisingWave Cloud
- Sign up and then sign in to the RisingWave Cloud.
- Create a cluster and ingest some streaming data. You can run the queries in the Fast Twitter events processing in the jumpstart queries to generate some mockup data.
- From the list of clusters, click Connect on the cluster you wish to connect to. Switch to a non-OAuth user and select the Parameter Only startup mode.Note down the connection details including host and port from the window that opens. You’ll need these connection details later.
Set up Cube Cloud
- In Cube Cloud, click Create Deployment to start.
- Click Next. In the next card, select CREATE to start a fresh deployment.
- Select Postgres, fill in the RisingWave connection details that you noted down earlier, and click Apply.
- When the connection is successful, you’ll be prompted to select tables to create a cube. After you select the tables, click Dimensions to customize the dimensions of the cube.
Let’s use the default settings for dimensions and specify hashtag as the primary key.
Click Review.
Click Confirm & Generate to generate the data model.
That's it! Cube will automatically build a GraphQL API for that table without you having to write a single line of code. When you navigate to the Overview section in Cube Cloud, you’ll see your APIs.
Then, you can securely query the GraphQL API from your frontend app or analytics tools instead of directly querying RisingWave. This separation of concerns makes your app more maintainable and scalable. As the materialized view in RisingWave always returns the latest value, you are guaranteed to get the latest value by making a request to the API endpoint.
curl
-H "Authorization: your_authorization_details_here"
-H "Content-Type: application/json"
--data-raw '{"query":"{cube{hot_hashtags{count}}}"}'
https://abcdefg.aws-us-east-1.cubecloudapp.dev/cubejs-api/graphql
{"data":
{"cube":
[
{"hot_hashtags":{"count":18}}
]
}
}
Additional Resources
In conclusion, the integration of RisingWave, a streaming database, and Cube, a semantic layer, offers a robust solution for event-driven applications and real-time data processing. With RisingWave's fresh analytical data delivered through real-time materialized views and Cube automatically generating GraphQL APIs, coding requirements are minimized. The seamless integration of Cube with various visualization tools further enhances the benefits for RisingWave users.
