Skip to lesson
Exit
Observability, Quality, Cost & Security1 / 3

2 min lesson

Tier datasets by what they feed

Talk this through in your own words: "A teammate proposes a single platform-wide SLA: every table fresh within one hour, page on any miss. Why is that a poor design and what would you propose instead?" Finish with the next move.

Step 1 of 3

Tier datasets by what they feednot everything is tier-1

Dataset class
Tier 1 - revenue / agent-training
Example
Billing facts, model-feed feature tables
Freshness target
Minutes to single-digit hours, completeness near 100%
On breach
Page on-call; block promotion of incomplete data
Dataset class
Tier 2 - core analytics
Example
Daily active users, retention marts
Freshness target
By the morning business day
On breach
Ticket + Slack to owner; degrade gracefully
Dataset class
Tier 3 - exploratory
Example
Ad-hoc tables, experiment scratch
Freshness target
Best-effort
On breach
No paging; visible in a dashboard only

Tiering is also a cost lever - tier-1 freshness is expensive, so you don't buy it for tables nobody depends on.

Learn more

Full explanation

Alert on symptoms, not just causes

Alert on symptoms, not just causeswhat the consumer actually feels

"Job failed" is a cause. "The gold table consumers read is now stale past its SLA" is a symptom. You want both, but the symptom alert is the one that maps to a broken promise - and it fires even when the job succeeded but produced nothing.

Cause-based (necessary, not enough)

Dagster run failed, Spark job OOM'd, connector errored.

Tells you what broke in the machinery.

Misses the case where everything ran but the output is wrong or empty.

Symptom-based (consumer-facing)

Gold table freshness lag exceeded its SLO.

Row count for the hour dropped to zero.

Fires on the thing a stakeholder would notice, regardless of cause.