1 min lesson
SLAs and how you'd alert on each
Choose two examples from the table in "SLAs and how you'd alert on each" and explain what each teaches you to do.
Step 1 of 3
SLAs and how you'd alert on eachfreshness and completeness are the two that matter
- SLA
- Freshness
- Definition
- Silver lags bronze by < N minutes
- Alert signal
- Max event-time watermark vs now exceeds threshold
- SLA
- Completeness
- Definition
- Daily event count within expected band
- Alert signal
- Volume anomaly vs trailing baseline (per source)
- SLA
- Lag
- Definition
- Consumer keeps up with the stream
- Alert signal
- Kafka consumer lag rising / not draining
- SLA
- Schema health
- Definition
- No rejected/quarantined spike
- Alert signal
- Dead-letter rate above baseline
| SLA | Definition | Alert signal |
|---|---|---|
| Freshness | Silver lags bronze by < N minutes | Max event-time watermark vs now exceeds threshold |
| Completeness | Daily event count within expected band | Volume anomaly vs trailing baseline (per source) |
| Lag | Consumer keeps up with the stream | Kafka consumer lag rising / not draining |
| Schema health | No rejected/quarantined spike | Dead-letter rate above baseline |
Tie every SLA to a concrete metric you can alert on - vague 'monitor it' answers don't land.
Learn more
Full explanation
Failure modes and recovery
Failure modes and recoverythe part that proves you've run one of these
Watermarks bound how late you'll wait in silver
Idempotent MERGE on event id collapses duplicates
Bronze retains raw, so a wider replay is always possible
Registry rejects breaking changes at the producer
Bronze permissive, silver strict - one table degrades, not all
Quarantine bad rows, alert, fix, replay from bronze
Stream buffers the spike; transforms drain at their pace
Autoscale consumers up to the partition ceiling
Alert when lag stops draining, not on every blip
Ingest keeps landing bronze independently
Backfill silver/gold from bronze once recovered
No data lost because raw is durable and replayable
“Client events hit a thin ingest gateway that stamps a server timestamp and an event id, then publish to Kafka topics partitioned for our peak - Kafka is the durable buffer so a client spike never touches downstream compute. Structured Streaming appends to bronze Delta, partitioned by event date, with an idempotent MERGE on the event id so replays don't double-count. Bronze→silver dedups, enforces the registered schema and uses watermarks for late events; malformed rows go to a dead-letter topic I can replay after a fix. Gold and the agent feature tables are derived from silver and fully rebuildable. I'd SLA on freshness and completeness, alerting on watermark lag and a volume anomaly per source.”
Close the design by naming what you'd cut for v1 and what you'd add at 10x. "For v1 I'd ship gateway → Kafka → bronze → silver with idempotent writes and a dead-letter path and defer liquid clustering and the feature store until the access patterns are real." Volunteering the staging plan signals the pragmatic, ship-fast judgment the role is filtering for, not gold-plating.