1 min lesson
p50 versus p99: design for the tail
Put yourself in this case: "Your service shows p50 of 120 ms but p99 of 2.5 s during peak hours. Where do you look first and why not the application code?" Give the clearest next step.
Step 1 of 2
p50 versus p99: design for the tailaverages lie at scale
At 1M+ DAU, the p99 is not a rare edge - it's tens of thousands of requests an hour and it's disproportionately your most engaged users hitting it repeatedly. A great median with an ugly tail still feels broken.
- Tail cause
- Queueing under load
- Symptom
- p99 explodes while p50 looks fine
- Mitigation
- Autoscale on the right signal; shed/queue with limits; add headroom
- Tail cause
- Cold starts
- Symptom
- First request to a scaled-up pod is slow
- Mitigation
- Pre-warm, keep a warm pool, scale earlier on leading signals
- Tail cause
- Cross-region hop
- Symptom
- A subset of users always slow
- Mitigation
- Regional pinning so data and compute sit near the user
- Tail cause
- GC / noisy neighbor
- Symptom
- Random slow requests, no pattern
- Mitigation
- Right-size, isolate, hedge with a second request on timeout
| Tail cause | Symptom | Mitigation |
|---|---|---|
| Queueing under load | p99 explodes while p50 looks fine | Autoscale on the right signal; shed/queue with limits; add headroom |
| Cold starts | First request to a scaled-up pod is slow | Pre-warm, keep a warm pool, scale earlier on leading signals |
| Cross-region hop | A subset of users always slow | Regional pinning so data and compute sit near the user |
| GC / noisy neighbor | Random slow requests, no pattern | Right-size, isolate, hedge with a second request on timeout |
Tail latency is usually a systems problem (queueing, placement), not slow code.