Hivemind Technologies: Building Streaming Medallion Architectures with RisingWave

Hivemind Technologies is a Germany-based data platform consultancy specializing in real-time data architectures. They chose RisingWave as the foundation for streaming Medallion architectures, delivering real-time data pipelines for enterprise clients.

Streaming Layers

>50%

Client Cost Reduction

SQL

Replaced ETL Scripts

~1000x

Faster Data

The Challenge

Why did Hivemind choose RisingWave for enterprise streaming projects?

Hivemind's enterprise clients were stuck on batch ETL pipelines that introduced hours of delay and required complex script stacks to maintain. They evaluated Apache Flink, Kafka Streams, and Spark Structured Streaming for their clients' real-time data needs.

Flink proved too heavyweight, requiring dedicated JVM expertise and complex cluster management. Kafka Streams was too low-level, demanding custom Java application development for every transformation. Spark Structured Streaming's micro-batch latency fundamentally limited real-time use cases. RisingWave's PostgreSQL-compatible SQL and built-in storage made it the ideal foundation for consultancy-led implementations where time-to-value matters.

The Solution

How does Hivemind implement the streaming Medallion architecture?

The traditional Medallion architecture (Bronze, Silver, Gold layers) typically runs on batch tools like Apache Spark. Hivemind's innovation was making the entire architecture stream-native with RisingWave — every transformation happens continuously in real-time using SQL materialized views instead of scheduled batch jobs.

Bronze Layer

Raw data ingestion from Kafka, MQTT, and CDC sources. No disk landing, no pre-processing -- every data point is immediately available in the streaming pipeline.

Silver Layer

Real-time cleaning and standardization using SQL materialized views. Unify field names, convert units, filter invalid data, and enrich with dimension tables.

Gold Layer

Real-time aggregated metrics using windowed materialized views. Serves dashboards, Iceberg tables, Kafka topics, and alerting systems simultaneously.

-- Bronze: Raw ingestion
CREATE SOURCE sensor_raw FROM KAFKA (...)
FORMAT PLAIN ENCODE JSON;

-- Silver: Clean and standardize
CREATE MATERIALIZED VIEW sensor_cleaned AS
SELECT device_id, standardized_value, location, event_time
FROM sensor_raw WHERE value IS NOT NULL;

-- Gold: Real-time metrics
CREATE MATERIALIZED VIEW device_health AS
SELECT location, AVG(value), COUNT(*) FILTER (WHERE value > threshold)
FROM TUMBLE(sensor_cleaned, event_time, INTERVAL '5 MINUTES')
GROUP BY location, window_start;

Results

What results do Hivemind's clients achieve?

Enterprise clients like Siemens see data latency drop from hours to seconds, infrastructure costs fall by over 50%, and cleaning logic simplified from complex script stacks to readable SQL. Business teams gain direct access to real-time data instead of waiting for next-day reports.

Metric	Traditional Batch	Streaming Medallion
Data latency	Hours (overnight batch)	Seconds
Transformation logic	Complex Spark/Airflow scripts	SQL materialized views
Infrastructure	Scheduling clusters + intermediate storage	Single streaming platform
Cost	High (dedicated batch compute)	>50% reduction
Maintenance	High (scripts, schedulers, landing zones)	Low (SQL, no scheduling)
Business access	Next-day reports	Real-time views and dashboards

Build Your Streaming Architecture

Replace batch ETL with real-time streaming Medallion pipelines.

Build Your Streaming Architecture →

Read the full Siemens case study →