Global Manufacturing Leader and Risingwave: Revolutionizing Production With Advanced Stream Processing Solutions

Company Background

Company A is a publicly traded global frontrunner in the manufacturing sector, renowned for its expertise in consumer and commercial LCD display technologies. Boasting a dedicated team of nearly 10,000 professionals worldwide, a notable 60% of whom are R&D engineers, the company underscores its profound commitment to groundbreaking innovation.

With a staggering sales record of almost 1 billion LCD devices globally, Company A's mission is a testament to its ambition and drive. As a top-tier global LCD display technology provider, they have expanded their product range to include LCD mainboards, interactive flat panels, medical devices, and other advanced intelligent hardware. At the heart of Company A's brand lies a commitment to collaborate with businesses that share its passion for innovation and unparalleled quality. This principle has enabled Company A to establish influential partnerships with major global companies, leveraging the best of these collaborations to deliver bespoke products and solutions tailored to the unique needs of their clientele.

The Landscape Before RisingWave

Company A placed its trust in real-time data analysis, leaning heavily on a commercial stream processing system, System X. Over three years, this platform integrated CDC streams from various databases (PostgreSQL, MySQL, Oracle), channeling output into Kafka. This setup was pivotal for their downstream analysis and the creation of real-time dashboards.

But it wasn’t a seamless journey. Company A grappled with several significant issues:

Stability: The system was plagued with various stability concerns. These ranged from cluster crashes due to internal errors in System X when incorporating new use-cases, to out-of-memory (OOM) incidents. The recovery process was arduous, demanding a complete recomputation because System X was exclusively in-memory. Given the nature of Company A's workloads, which frequently involved over 10 streaming inner joins in a single query, they resorted to procuring machines boasting 1TB of memory.
Observability: Company A found it taxing to monitor their streaming job effectively. Issues arose in gauging source throughput, tracking System X creation progress, and keeping tabs on CPU and Memory usage through System X.
Support: Assistance from the vendor was less than satisfactory. Responses to feature requests or resolution of existing problems were slow to come.

Why RisingWave?

Discontented with their current system, Company A sought a robust alternative. RisingWave caught their attention. The advanced distributed streaming database offered a suite of features that filled the gaps left by their previous system:

Reliability: RisingWave boasts a persistent and consistent checkpointing system. This not only boosts the availability of streaming jobs—enabling immediate resumption post-recovery—but also simplifies maintenance. Engineers no longer have to fret over full recomputation issues, given the dependable checkpoint system.
Scalability: The platform adopts a decoupled compute-storage architecture. This facilitates effortless and efficient scalability. For instance, Company A can conveniently expand their computational resources depending on their workload, without impinging on storage resources.
Efficient Joins: RisingWave excels in offering stable multi-way streaming join support. Given Company A’s requirement to conduct streaming joins on tables from varied databases, often involving 10+ joins, there's a need for a system that can handle such intensity. RisingWave capably supports this many-join use case, offering unprecedented freshness that was elusive before.
Observability: RisingWave brings improved visibility to streaming jobs. Firstly, the platform offers a range of metrics through Grafana dashboards—perfect for continuous monitoring. Secondly, internal states of streaming jobs are SQL-queryable, a feature that's proven invaluable for debugging. Moreover, with streaming jobs fashioned as queryable materialized views and RisingWave's SQL being postgres-compatible, debugging SQL and data-related issues has never been simpler.
Support: The RisingWave team stands out with their commitment to customer support. They're consistently receptive to Company A's needs, assisting them with deployment and troubleshooting. Additionally, they're adept at rolling out high-priority feature enhancements and fixes promptly.

RisingWave in Production

Company A initiated a collaboration with the RisingWave team approximately one year ago, focusing on innovation-driven development. Initially, RisingWave's architecture was still evolving and lacked several crucial system integration features, restricting Company A to basic tests and sandbox environments.

In response to feedback and requirements from partners, including Company A, RisingWave enhanced its capabilities over several months. These advancements included the missing features, thereby meeting the demands of potential clientele.

Capitalizing on these developments, Company A promptly deployed an SRE for integration using Kubernetes. Furthermore, a specialist engineer verified RisingWave's performance under production conditions.

Today, RisingWave is integral to Company A's infrastructure, managing numerous materialized views and enhancing real-time dashboard capabilities. Its architecture efficiently manages complex database operations, such as 7-way and 11-way joins. The system's proficiency in processing data via debezium CDC and routing to downstream databases using JDBC sink stands out. Company A's technical team interacts with RisingWave daily via the DBeaver UI.

Looking forward, Company A intends to expand RisingWave's applications, venturing into new use-cases to maximize this collaborative potential.