1 min lesson
p50 is a vanity metric; p99 is the product
Recall the main items in "p50 is a vanity metric; p99 is the product", then connect each one to the work.
Step 1 of 2
p50 is a vanity metric; p99 is the product
A great median with a bad tail still feels broken, because the user remembers the stall, not the 200 fast completions before it. Tab prediction has a hard UX cliff: past roughly 100–150ms the suggestion arrives after the user has already typed past it and a late suggestion is worse than none - it's a distraction.
- Set the SLO on the tail (p99), not the median and treat a p99 regression as a product bug, not a perf nicety.
- Shed load gracefully: under pressure, drop to the small model or skip the completion rather than serving it late.
- A timed-out completion should silently no-op; never show a stale suggestion two keystrokes after it was relevant.
Don't optimize average latency and call it done. The interviewer is listening for whether you distinguish p50 from p99 and whether you know the typing-flow cliff exists. An answer that names the cliff - "a late tab completion is negative value" - separates product engineers from people who've only read the benchmark.
Learn more
Optional practice
Practice: p50 is a vanity metric; p99 is the product
QIn a tab-prediction design, why is optimizing p99 latency more important than optimizing the median?