It has been almost three years since we, RisingWave Labs, embarked on the journey of democratizing stream processing in the cloud era. We open-sourced the RisingWave database in April 2022 and launched the General Availability of RisingWave Cloud in July 2023. A lot of people have asked, what is the difference between open-source RisingWave and RisingWave Cloud? What more can I get on RisingWave Cloud? What is your next move on RisingWave Cloud? We have recently published a blog describing the roadmap of RisingWave. Now it is the time to answer these questions by looking at what the plan is ahead for RisingWave Cloud.
Please note that this roadmap could vary depending on the user feedback and customer requests. We would love to hear the feedback from you and understand how can we make RisingWave Cloud better.
Core objectives
The core objective of RisingWave Cloud has always been crystal clear: to democratize stream processing on the cloud. Powered by open-source RisingWave, customers can create their stream processing applications easily using familiar PostgreSQL-style SQL and PostgreSQL-compatible tools. However, RisingWave Cloud is much more than that. The core objectives of RisingWave Cloud can be broken down as follows.
- Bringing stream processing experience to the next level.
- Comprehensive management tools.
- Cost-efficient infrastructure solutions.
- Being adaptive to various user environments and cloud ecosystems.
Next we will introduce the detailed roadmap into three phases: short-term goals, mid-term goals, and long-term goals.
Short-term goals: (Second half of 2023)
The short-term goal of RisingWave Cloud is to provide users with a self-serving ready-to-use cloud product. Users can set up their RisingWave clusters easily on the cloud and start their stream processing journey. To achieve this goal, we have implemented many cool features in our SaaS product covering the functionalities of cluster management, web console, and monitoring. We will continue working in this direction and deliver the following features by the end of 2023.
- Launching developer tier service. We get constant feedback from our customers that a 7-day trial is not enough for their PoC testing, especially for individual developers. We are happy to promise that we will launch a new developer tier service, which will offer a standalone RisingWave cluster of \$99 per month.
- More connector wizards. Configuring sources in RisingWave is easy, but not that easy. You still have to fill in tens of parameters each time to figure out how to set up a connector. We have already launched the Confluent Cloud connector and the Apache Kafka connector in RisingWave Cloud to guide users on how to set up a connector by adding hints and explanations. We will add more connectors that have already been supported by the open-source RisingWave database.
- BYOC service on AWS and GCP: private preview. Without any question, Bring Your Own Cloud (BYOC) is an evolving trend for cloud services. In short, BYOC strikes the balance between the convenience of cloud service and the requirements of security compliance. As one of the most frequently requested features, we will launch BYOC in a private preview mode. Please contact us if you are interested in trying out BYOC. We will have a series of future blogs discussing the details of our BYOC designs.
- Multi-user support with basic RBAC. We will introduce an organization model in RisingWave Cloud, in which all users in the same organization can share the clusters and other resources directly. We will have some basic Role-Based Access Control (RBAC).
Mid-term goals: (First half of 2024)
We are never satisfied with only providing only the basic functionalities. The mid-term goal of RisingWave Cloud is to enlarge the scope of the product to be integrated with the modern cloud ecosystem, and drastically improve the user experience. Some detailed goals include:
- Azure support. We will support Azure as our underlying platform. Users will be able to launch their RisingWave clusters easily and connect with any other infrastructures and services in Azure.
- BYOC General Availability. By the middle of 2024, we will have the BYOC feature generally available across all platforms: AWS, GCP, and Azure. The whole package of BYOC mode will include a whole set of infrastructure being installed on users’ cloud environment, with the flexibility to plugin users’ existing infrastructures at different levels, and offer global SLA guarantees.
- UDF server support. UDF server is a unique feature supported by the open-source RisingWave. The UDF server is meant to interact with external systems and services aside from RisingWave, so we recommend users host UDF servers by themselves. However, we will still provide a simplified service so that users can use our Python and Java UDF easily only through our cloud.
- Cluster backup. RisingWave Cloud will backup the whole cluster regularly and the cluster can recover from the checkpoints. We will have this feature running the backend first by the end of 2023, and then open a management panel for users to manage all the versioned backups later in 2024.
- Cluster-level access control. Beyond basic role-based access control, we will push forward the boundary and introduce cluster-level access control in RisingWave Cloud. Different users will have different access to different clusters in the organization, and the administrator can grant access to different clusters for different roles and user groups.
- Developer API and CLI. RisingWave Cloud will publish a set of developer APIs and CLI tools to offer developers programmable access to their RisingWave Cloud accounts. Based on that, we will also launch a set of partnership programs for our partners to customize their deep integration with the RisingWave Cloud product.
- Integration with third-party monitoring tools. As a part of developer API, we will introduce a programmable framework so that users can connect their existing monitoring tools to track the RisingWave Cloud monitoring data via OpenTelemetry protocol.
Long-term goals: (Second half of 2024)
It is hard to predict the future, we don’t exactly know where the future will lead us, and we are open to taking user requests for our product roadmap. Nevertheless, we have a few features planned for long-term investigation.
- Multi-region deployment. We plan to support the multi-region deployment of RisingWave clusters to enhance the availability of RisingWave. This will significantly improve the robustness of online RisingWave services.
- Auto-scaling for stream processing. We will introduce we separate service to embrace the power of elasticity of cloud computing. To start, we will explore how to automatically scale the RisingWave compute nodes in the cloud smoothly, so that users will not need to worry about maintaining the RisingWave clusters and adjusting the parameters.
- Serverless compaction. Similarly, we will explore the same idea for compaction. Compaction is an important step in RisingWave state management and could be the bottleneck of the streaming pipelines. We will explore an auto-scaling policy and framework so that the compactor can scale to adapt to the floating workloads.
- RisingWave-native observability tool. Observability is one of the most vital component in modern cloud infrastructures. Although there are so many existing monitoring tools, which we plan to integrate as discussed, only RisingWave developers understand RisingWave the best and can develop the best tool that has deep integration with RisingWave and tell you the bottleneck of the system thoroughly. We are still in the early investigation and we plan to upgrade our RisingWave-native observability tool only available on the cloud.
- Attribute level access control. We will push forward the ability of the RBAC on RisingWave Cloud, to provide users with more convenient and fine-grained access control on the attribute level.
RisingWave Cloud cannot go this far without the support of our customers and our community members. We are happy to share our plans on how we democratize stream processing. To stay tuned with RisingWave and RisingWave Cloud, check out our website, subscribe to our newsletter, follow us on social media, join our community slack, and talk to us directly!
