1 min lesson
Change-point detection over static thresholds
Use "Change-point detection over static thresholds" to tell the cases apart, then choose a response for each one.
Step 1 of 2
Change-point detection over static thresholds
A fixed threshold like "alert if p95 > 4s" breaks both ways: it's deaf to a regression that moves p95 from 800ms to 1.6s and it screams every time legitimate traffic shifts. Change-point detection asks a better question - did the distribution's behavior shift at some point in time, relative to its own recent history?
Static threshold
"Alert if error rate > 1%."
Misses a 0.2% → 0.6% tripling that's plainly a regression.
Needs constant hand-tuning per metric and segment.
Change-point / anomaly
"Alert when the series departs its own recent baseline."
Catches relative shifts at any absolute level.
Adapts as baselines drift with traffic and seasonality.