1 min lesson
The DLQ is a debugging and recovery tool, not a graveyard
Work through the cases in "The DLQ is a debugging and recovery tool, not a graveyard", pairing each signal with the move that fits.
Step 1 of 2
The DLQ is a debugging and recovery tool, not a graveyardwhat it buys an EM running on-call
A dead-letter queue isolates the events your pipeline couldn't deliver so they don't block everything behind them. The events sit there, inspectable, while the main flow keeps moving. That separation is what lets you fix a root cause calmly and then replay.
Poison messages leave the hot path.
One broken consumer can't stall the queue for everyone.
DLQ depth is a leading incident signal.
Each message carries why it failed - fast triage.
Fix the bug, then re-drive the DLQ.
Idempotent consumers make replay safe to repeat.
When you sketch the retry path, alert on two signals out loud: DLQ depth climbing and retry rate spiking. Both are leading indicators - they move before customer-facing success rate craters. Saying "these page me before the SLO breaks" reframes you from someone who builds pipelines to someone who operates them, which is the whole point of an EM round.
A DLQ with no alarm and no replay tooling is just a slow data leak. If the answer to "what happens after the DLQ?" is "we look at it eventually," you've described losing events with extra steps. Always pair the DLQ with an alert on its depth and a documented replay path.