Detection Engineering

Sigma Rules on Streaming Data

Translate Sigma detection rules into SQL and evaluate them continuously over Kafka log streams. RisingWave runs hundreds of detection rules in parallel as materialized views, surfacing matches in real time without waiting for a SIEM to index logs.

Start Free Threat Detection

Real-Time

Rule Evaluation

Sigma rules translated to SQL evaluate against every log event as it arrives in the Kafka stream, not after a SIEM indexing cycle

SQL

Rule Translation

Sigma condition fields, aggregations, and timeframes map directly to SQL WHERE clauses, COUNT windows, and GROUP BY expressions

Parallel

Rule Execution

Hundreds of Sigma rules run simultaneously as independent SQL materialized views over the same source stream without per-rule infrastructure

PostgreSQL

Output Interface

Detection matches surface as rows in a materialized view queryable via PostgreSQL protocol, ready for SIEM ingestion or SOAR automation

Why Streaming Evaluation

Why translate Sigma rules to SQL for streaming evaluation?

Sigma rules are written for SIEM search languages like SPL or KQL, which operate on indexed log data. Indexing introduces minutes of latency between a log event and rule evaluation. Translating Sigma rules to SQL and running them in RisingWave over Kafka streams evaluates the same detection logic against every event as it arrives, eliminating the indexing latency entirely.

Factor	SIEM-Based Sigma	RisingWave
Evaluation Latency	Minutes (after SIEM index)	Milliseconds (per-event)
Rule Language	SPL, KQL, Sigma YAML	Standard SQL
Rule Throughput	Sequential SIEM search	Parallel materialized views
Infrastructure	Log shipper + SIEM cluster	Kafka + RisingWave

→Evaluate the same Sigma detection logic in milliseconds instead of waiting for the SIEM index cycle
→Run hundreds of detection rules in parallel as independent SQL views over a shared Kafka source stream
→Reduce SIEM costs by filtering and pre-aggregating log events before they reach the indexing layer
→Extend detection coverage with custom SQL rules that complement Sigma rule libraries

Use Cases

What Sigma rule categories translate well to streaming SQL?

Sigma rules that detect event sequences, frequency thresholds, or cross-source correlations benefit most from streaming evaluation. These patterns require stateful computation that SQL window aggregations and stream-table joins handle natively, without the stateless search approach that SIEM tools use.

Process Execution Rules

Sigma rules detecting suspicious process execution patterns translate to SQL filters over endpoint telemetry streams from Kafka. Evaluate parent-child process relationships, suspicious command-line arguments, and execution paths as rows arrive without log indexing

Authentication Failure Rules

Brute force and credential stuffing Sigma rules map to SQL COUNT aggregations with OVER time windows. Run authentication failure threshold rules continuously over Active Directory and Okta event streams, triggering on the exact event that crosses the threshold

Network Connection Rules

Sigma rules detecting outbound connections to suspicious ports or IP ranges translate to SQL filters with stream-table joins against threat intelligence tables updated in real time via CDC. New threat intel entries take effect within seconds

Lateral Movement Sequences

Multi-event Sigma rules detecting lateral movement sequences translate to SQL session window aggregations that correlate authentication events across hosts within a time bound, surfacing the pattern without waiting for all events to be indexed

How It Works

How do you run Sigma rules over Kafka streams in RisingWave?

Each Sigma rule translates to a SQL SELECT statement or materialized view in RisingWave. The Sigma condition block becomes a SQL WHERE clause. Time-bound aggregations in Sigma become SQL window functions. The Sigma detection timeframe becomes a window interval. Once translated, the rule runs continuously over the Kafka source stream as an incremental view.

→Create a Kafka source in RisingWave for each log category (endpoint, auth, network) using SQL CREATE SOURCE statements
→Translate each Sigma rule condition to a SQL SELECT: map detection fields to column filters, timeframes to window intervals, and count thresholds to HAVING clauses
→Create a SQL materialized view for each translated rule, or combine related rules into a single view with a UNION to reduce resource usage
→Join the detection stream against threat intelligence and asset inventory tables via stream-table joins to enrich matches with context
→Query the detection materialized view from your SOAR platform or alert router via the PostgreSQL interface to act on matches

Frequently Asked Questions

How do I translate a Sigma rule into SQL for streaming evaluation?

Map the Sigma detection block fields to SQL column filters in a WHERE clause. Translate the timeframe field to a SQL tumbling or hopping window interval. Convert count thresholds in Sigma aggregations to SQL HAVING clauses with COUNT. The Sigma condition logic maps to SQL AND/OR expressions. Create the resulting SELECT as a RisingWave materialized view over your Kafka log source, and it will evaluate continuously as events arrive.

Can I run the full Sigma rule library over Kafka streams?

How does streaming Sigma rule evaluation compare to running Sigma rules in a SIEM?

SIEM-based Sigma evaluation runs rules as search queries on indexed log data, introducing indexing latency of minutes or more. RisingWave evaluates translated SQL rules against each log event as it arrives in the Kafka stream, reducing detection latency to milliseconds. Running rules as materialized views also means detection state is maintained incrementally rather than recomputed from scratch on each search execution.

Can I mix Sigma-translated rules with custom SQL detection rules?

Evaluate detection rules at stream speed, not SIEM speed

Translate Sigma rules to SQL and run them continuously over Kafka log streams without waiting for a SIEM to index your logs.

Start Free