High Availability in Stream Processing: Active-Active and Failover Patterns
Streaming workload costs come from three sources: compute (processing nodes), storage (state and checkpoints), and network (data transfer). Optimizing each reduces total cost without sacrificing performance.
Cost Breakdown
| Component | Traditional (Flink + RocksDB) | Disaggregated (RisingWave) |
| Compute | Over-provisioned for state | Right-sized for processing |
| Storage | Local SSD ($0.10+/GB) | S3 ($0.023/GB) |
| State | On compute nodes | On S3 (independent) |
| Scaling | Scale compute = scale state | Scale independently |
Optimization Tips
- Use S3 for state: 5-10x cheaper than local SSD
- Right-size compute: Disaggregated state means compute only handles processing
- Tune checkpoint interval: Longer intervals = less checkpoint I/O overhead
- Filter early: Drop unneeded events before processing to reduce compute
- Use Iceberg tiering: Hot data on S3 Standard, cold data on S3 Glacier
Frequently Asked Questions
How much does stream processing cost?
Depends on throughput, state size, and tool choice. RisingWave on modest hardware (4 cores, 16GB RAM) + S3 handles most small-to-medium workloads for $100-300/month. Flink clusters typically cost $500-2000/month for equivalent workloads.
