Skip to lesson
Exit
Order-to-Cash, Deep1 / 2

1 min lesson

Failure handling that preserves controls

Recall the main items in "Failure handling that preserves controls", then connect each one to the work.

Step 1 of 2

Failure handling that preserves controlsDLQ, replay, manual correction

Events will fail to process. The question is whether a failure quietly drops a dollar or lands somewhere recoverable. In finance, manual correction is legitimate, but it must keep segregation of duties and leave evidence.

  • Dead-letter queue. When an event exhausts its retries, route it to a DLQ for isolation and alerting - never drop it. A DLQ that is filling up is a P1 because it is unrecognized revenue or unapplied cash.
  • Replay. Because raw events are append-only and idempotent, you can re-run processing over a window once the bug is fixed and the idempotency keys prevent the replay from double-counting.
  • Manual-correction workflow. When a human must intervene, do it through a tracked workflow with maker-checker approval and an immutable record of who changed what and why - not a direct edit to a production table.