Stream Processing
Difficulty: Hard | Topic #25
What to Learn
Kafka as an event log, Flink/Kinesis for stateful stream processing, windowed aggregations (tumbling, sliding, session windows), exactly-once semantics, watermarks for out-of-order events.
Resources
- Hello Interview: Flink Deep Dive ↗
- ByteByteGo: Why is Kafka Fast? ↗
- System Design: Kafka, Flink, Spark, Exactly-Once Semantics ↗
Covered by Problems
| Problem | Difficulty | Link |
|---|---|---|
| Live Comments (FB Live) | Medium | → |
| Ad Click Aggregator | Hard | → |
| Metrics Monitoring System | Hard | → |
| Search Engine / FB Post Search | Hard | → |
Key Concepts to Master
- Tumbling vs sliding vs session windows and their memory implications
- Watermarks for handling late-arriving events
- Exactly-once processing via checkpointing in Flink
- Stateful operators and RocksDB state backend
- Kafka as the input source and the at-least-once vs exactly-once contract