Message Queues & Async Processing
Difficulty: Medium | Topic #14
What to Learn
Producer-consumer pattern, Kafka (topics, partitions, consumer groups), SQS, at-least-once vs exactly-once semantics, dead letter queues, backpressure, when queues beat synchronous calls.
Resources
Covered by Problems
| Problem | Difficulty | Link |
|---|---|---|
| Online Judge (LeetCode) | Medium | → |
| Messaging App (WhatsApp) | Medium | → |
| Video Streaming (YouTube) | Medium | → |
| Live Comments (FB Live) | Medium | → |
| Job Scheduler | Medium | → |
| Payment System | Hard | → |
| Ad Click Aggregator | Hard | → |
Key Concepts to Master
- Kafka topic/partition/consumer group model and why it enables parallelism
- At-least-once vs exactly-once semantics and idempotent consumers
- Dead letter queues for poison messages
- Backpressure mechanisms to protect downstream services
- When to use a queue (decoupling, buffering) vs direct RPC