1 min lesson
Detecting drift before it costs a quarter
Choose two examples from the table in "Detecting drift before it costs a quarter" and explain what each teaches you to do.
Step 1 of 2
Detecting drift before it costs a quarterthe silent failure mode
Pipelines rot quietly. A data provider changes its schema and match rate slips from 75% to 60% over three weeks; nobody notices until pipeline dries up. Drift detection means the metric watches itself.
- Drift signal
- Match rate drops week over week
- Likely cause
- Provider schema or coverage change
- What the loop should do
- Alert; fall back to next provider in the waterfall
- Drift signal
- Qualified rate climbs but conversion falls
- Likely cause
- Scoring inflated; threshold too loose
- What the loop should do
- Re-fit weights against recent outcomes
- Drift signal
- Routing SLA hit rate decays
- Likely cause
- Volume spike or owner capacity change
- What the loop should do
- Re-balance round-robin; alert on backlog
- Drift signal
- Enrichment cost per lead rises
- Likely cause
- Cheap provider missing more; falling to pricey one
- What the loop should do
- Re-order the waterfall by cost-adjusted hit rate
| Drift signal | Likely cause | What the loop should do |
|---|---|---|
| Match rate drops week over week | Provider schema or coverage change | Alert; fall back to next provider in the waterfall |
| Qualified rate climbs but conversion falls | Scoring inflated; threshold too loose | Re-fit weights against recent outcomes |
| Routing SLA hit rate decays | Volume spike or owner capacity change | Re-balance round-robin; alert on backlog |
| Enrichment cost per lead rises | Cheap provider missing more; falling to pricey one | Re-order the waterfall by cost-adjusted hit rate |
Each signal maps to an automated response, not a quarterly review.
Learn more
Full explanation
Experiments with honest readouts
Experiments with honest readoutsA/B on routing and messaging
When you A/B a routing rule or an outreach variant, the discipline is the same as any experiment. Randomize at the lead or account level, pre-register the metric and resist peeking until you have the sample to call it. A directional read stated as a directional read beats a false certainty.
- Randomize at the right unit (account, not lead, when reps work whole accounts) to avoid contamination.
- Pre-commit to the primary metric and the minimum sample before you launch, so you can't retrofit a winner.
- Report the effect with its uncertainty and call out when the sample is too small to conclude anything.