sFlow's continuous wire-speed packet sampling lets RisingWave detect volumetric floods within seconds. By streaming sFlow records and HTTP access logs into Kafka, you can run SQL window aggregations directly on live traffic data, alerting on anomalous byte volumes or request rates before an attack overwhelms your infrastructure.
What is sFlow and how does it differ from NetFlow for DDoS detection?
sFlow is a packet sampling protocol designed for wire-speed network monitoring. Every network device samples 1-in-N packets and exports those samples immediately, producing a continuous stream of traffic data with sub-second to low-second latency. Because sFlow exports samples as they are captured rather than waiting to close flow records, you get near-real-time visibility into traffic patterns.
NetFlow works differently. Routers and switches accumulate per-flow records in a cache and export them when flows expire or on a timer. The default export interval is typically 30 to 60 seconds, and flows may stay in cache longer before being flushed. This means NetFlow-based DDoS detection carries a structural latency floor of at least 30 to 60 seconds before the data even arrives at your analytics system. NetFlow is well suited to volumetric trend analysis and capacity planning over longer windows, but that export interval makes it a poor fit for real-time attack response.
For DDoS detection, the protocol choice determines how quickly you can observe an attack. sFlow gives you traffic data within seconds; NetFlow gives you data after the export interval plus whatever processing time your analytics pipeline adds.
What DDoS attack types are detectable with streaming SQL?
Three broad attack categories map well to streaming SQL queries against sFlow and HTTP log data.
Volumetric floods saturate bandwidth or packet-per-second capacity. These are visible in sFlow as a sudden spike in bytes or packets destined for a single IP address. A SQL window aggregation over a 10-second tumbling window can compare current traffic volume against a rolling baseline and fire an alert when the ratio exceeds a threshold.
Amplification attacks exploit protocols like DNS and NTP, where a small spoofed request triggers a large response. In sFlow data, these appear as a high ratio of response bytes to request bytes for a given destination. You can detect them by joining sFlow records for outbound requests against inbound responses and computing the amplification factor per source IP in a sliding window.
Application-layer HTTP floods do not generate unusual byte volumes at the network level because each individual request is small. Instead, they overwhelm server-side resources by sending thousands of requests per second to specific endpoints. These attacks are best detected from HTTP access logs streamed via Kafka, where you can count requests per source IP per endpoint over a one-minute window and compare against a per-endpoint baseline.
How to detect volumetric floods with SQL window aggregations
The core pattern is straightforward: sum the bytes arriving at each destination IP over a fixed tumbling window, then compare that sum against a long-term average.
RisingWave accepts sFlow data as a Kafka source. Each message contains fields including the source IP, destination IP, bytes sampled, and the sampling rate. Because sFlow captures only 1-in-N packets, you multiply the sampled byte count by the sampling rate to estimate the true volume.
-- Volumetric flood detection: estimated bytes per destination IP per 10-second window
CREATE MATERIALIZED VIEW volumetric_flood_alerts AS
SELECT
window_start,
window_end,
dst_ip,
SUM(bytes_sampled * sampling_rate) AS estimated_bytes,
COUNT(*) AS sample_count
FROM TUMBLE(
sflow_records,
ingestion_time,
INTERVAL '10 seconds'
)
GROUP BY
window_start,
window_end,
dst_ip
HAVING SUM(bytes_sampled * sampling_rate) > 1000000000; -- 1 GB threshold per 10s window
This materialized view is continuously updated as new sFlow records arrive. Any destination IP accumulating more than 1 GB of estimated traffic in a 10-second window appears in the view immediately. Downstream alerting systems can poll this view or subscribe to its changelog via Kafka to trigger automated mitigation.
To make the threshold dynamic rather than static, you can create a second materialized view that maintains a rolling 24-hour average per destination IP, then join it against the 10-second window results to flag windows that exceed, say, five times the baseline average.
Detecting application-layer HTTP floods
HTTP flood detection follows the same pattern but operates on a different source: HTTP access logs streamed from your web servers via Kafka.
Each log line contains the source IP, the requested URL path, the HTTP method, the response status, and a timestamp. RisingWave consumes these as a Kafka source and lets you aggregate them in SQL.
-- HTTP flood detection: requests per source IP per endpoint per minute
CREATE MATERIALIZED VIEW http_flood_alerts AS
SELECT
window_start,
window_end,
src_ip,
endpoint,
COUNT(*) AS request_count,
COUNT(DISTINCT request_id) AS unique_requests
FROM TUMBLE(
http_access_logs,
request_time,
INTERVAL '1 minute'
)
GROUP BY
window_start,
window_end,
src_ip,
endpoint
HAVING COUNT(*) > 500; -- 500 requests per IP per endpoint per minute
Because the detection logic runs inside RisingWave as an incrementally maintained materialized view, the result set updates within seconds of new log records arriving. You can combine this with a stream-table join against a whitelist of known IPs or an allowlist of high-traffic service accounts to reduce false positives.
For distributed HTTP floods that spread traffic across many source IPs (often called low-and-slow floods), you can aggregate at the endpoint level rather than the per-IP level to detect unusual total request rates against a specific path.
Step-by-step: building a DDoS detection pipeline
1. Define Kafka sources for sFlow and HTTP logs.
Your sFlow collector (such as sflowtool or GoFlow2) serializes sFlow datagrams and publishes them to a Kafka topic. HTTP access logs are forwarded to a separate Kafka topic by Filebeat, Fluentd, or a similar log shipper.
-- sFlow records source
CREATE SOURCE sflow_records (
src_ip VARCHAR,
dst_ip VARCHAR,
src_port INT,
dst_port INT,
protocol INT,
bytes_sampled BIGINT,
packets_sampled INT,
sampling_rate INT,
ingestion_time TIMESTAMPTZ
) WITH (
connector = 'kafka',
topic = 'sflow-records',
properties.bootstrap.server = 'kafka:9092',
scan.startup.mode = 'latest'
) FORMAT PLAIN ENCODE JSON;
-- HTTP access logs source
CREATE SOURCE http_access_logs (
src_ip VARCHAR,
endpoint VARCHAR,
method VARCHAR,
status_code INT,
response_bytes BIGINT,
request_id VARCHAR,
request_time TIMESTAMPTZ
) WITH (
connector = 'kafka',
topic = 'http-access-logs',
properties.bootstrap.server = 'kafka:9092',
scan.startup.mode = 'latest'
) FORMAT PLAIN ENCODE JSON;
2. Create materialized views for anomaly detection. Use the SQL definitions from the previous sections. RisingWave evaluates each arriving record against the window aggregation incrementally, updating the materialized view output as data flows in.
3. Expose results to alerting systems. Query the materialized views directly from Grafana or another dashboard tool, or create a Kafka sink from each detection view to publish alerts downstream. A Prometheus integration lets you feed alert counts into existing incident management workflows.
4. Add a baseline comparison view. Maintain a separate materialized view over a 1-hour or 24-hour window for each destination IP or endpoint, and join the short-window view against it at query time to compute the deviation ratio. This makes thresholds adaptive to normal traffic patterns.
How does this compare to traditional NetFlow-based DDoS detection?
Traditional DDoS detection pipelines built on NetFlow typically route flow records through a collector, load them into a time-series database or SIEM, and run batch queries on a polling interval. The total detection latency combines the NetFlow export interval (30 to 60 seconds by default), the collection and normalization time, and the query polling interval.
RisingWave removes the analytics delay from that equation. Once data arrives, the materialized view updates are computed incrementally and continuously, so there is no separate query polling overhead. For sFlow data, where the export latency is already sub-second, this means you can achieve detection latencies measured in seconds from packet-hit to alert.
For NetFlow, RisingWave improves the analytics half of the pipeline but cannot change the fact that the flow records themselves arrive 30 to 60 seconds after the traffic occurs. If your current NetFlow pipeline takes 90 seconds from traffic to alert (60-second export interval plus 30-second query polling), using RisingWave reduces that to closer to 60 to 65 seconds. That is a meaningful improvement for large-scale volumetric detection, but it is not a substitute for sFlow when sub-minute detection is required.
The practical recommendation is to use sFlow for real-time detection and automated mitigation triggers, and use NetFlow for longer-horizon analysis such as traffic trending, capacity planning, and post-incident forensics where its richer flow metadata is valuable.
FAQ
Can RisingWave handle high-volume sFlow streams in production?
Yes. RisingWave is built for high-throughput stream processing and scales horizontally across multiple compute nodes. sFlow sampling rates are typically configured at 1-in-1000 or 1-in-10000 for high-speed interfaces, which keeps the ingestion rate manageable even for multi-Gbps links. RisingWave's Kafka connector supports multiple consumer partitions for parallel ingestion.
How do I avoid false positives during legitimate traffic spikes?
Use dynamic thresholds derived from a rolling baseline materialized view rather than fixed byte counts. Join the short-window detection view against a per-IP or per-endpoint baseline and flag windows that exceed the baseline by a configurable multiplier. Combine this with a whitelist stream-table join for known high-traffic services.
Does RisingWave guarantee exactly-once processing for sFlow records?
Yes. RisingWave provides exactly-once semantics for Kafka sources by tracking Kafka offsets durably and recovering from checkpoints on failure. This means sFlow records are counted exactly once in aggregations even during node restarts or failures.
Can I use RisingWave for both network-layer and application-layer DDoS detection simultaneously?
Yes. You can define multiple sources and materialized views in the same RisingWave instance, covering sFlow for volumetric and amplification attacks and HTTP access logs for application-layer floods. Each materialized view maintains its own incremental state and updates independently as records arrive on the respective Kafka topics.
Real-time DDoS detection stops attacks before they cause downtime. RisingWave brings SQL-native stream processing to sFlow and HTTP log data, turning continuous packet samples into live detection views that update within seconds of traffic changes.
Try RisingWave Cloud free and connect your Kafka sFlow stream to start building detection pipelines today.
