2 min lesson
What good detection looks like on a lakehouse
Rebuild the parts of "What good detection looks like on a lakehouse", then say why each one matters.
Step 1 of 2
- Freshness
- Asset hasn't materialized within its SLO, which catches stalled ingestion before consumers do
- Volume
- Row count drifts outside the expected band, which catches a dropped source or a partial load
- Distribution
- A column's stats shift, with nulls spiking or cardinality dropping, which catches silent schema and quality rot
- Lineage
- Knowing which gold tables a bad bronze partition touched turns containment from guesswork into a query
These are the checks that let you say "we caught it before the customer did."
"A third-party source silently changed a field's type and our bronze load coerced it to null. The first I knew was a finance lead asking why revenue dropped 8% overnight, which told me the real failure was that we had no distribution check and no schema gate. I quarantined the bad partitions, rolled the gold revenue table back to the prior day's snapshot and posted a plain-English impact note: which dashboards, which dates, ETA. The lasting fix wasn't the backfill. It was a schema-registry gate on ingestion plus null-rate checks that now fail the asset closed before it can poison gold. We haven't had a silent type-change leak since."
Keep it blameless and own the gap. "The vendor broke their schema" with no mention that your platform had no gate at the bronze boundary reads as passing blame. "I stayed up all night fixing rows" casts you as the single point of failure. Say what the system lacked, and that you added it.