1 min lesson
Latency is a distribution, not an average
Give a plain answer to "Why track latency as a distribution rather than an average?" Then ground it in one lesson detail.
Step 1 of 2
Latency is a distribution, not an averagethe tail is what users feel
An average hides the requests that actually hurt. If p50 is 40ms but p99 is 600ms, one in a hundred completions feels broken and a developer hits that tail many times an hour. Manage latency as percentiles and write SLAs against the tail.
Reading a latency distribution
- p50 (median)
- The typical experience; good for capacity math, useless as an SLA
- p95
- Where most SLAs live; the bad-but-common case
- p99 / p99.9
- The tail - rare per request, frequent across a day of use; this is what users remember
Batching that lifts throughput often fattens the tail - watch p99, not just the mean.