1 min lesson
Techniques to buy milliseconds back
Walk through the important items in "Techniques to buy milliseconds back" and give the practical point of each.
Step 1 of 2
Techniques to buy milliseconds backwhere the savings actually are
- Connection reuse and HTTP/2 or HTTP/3 kill repeated handshakes, often the cheapest big win for chatty clients.
- Regional pinning: put the user, their data and their compute in one region so the network segment stays small.
- Cache the cacheable at the edge so those requests never pay backend or network cost at all.
- Shed work to the edge: do auth, rate-limit and routing decisions near the user and short-circuit early.
- Hedged requests for the tail: on a slow outlier, fire a second request and take the first to return. You spend a little compute to cut p99.
Budget, then spend
The discipline is to set the budget first, instrument each segment and then make spend/save decisions against data. "We added 30 ms of edge auth to remove a 90 ms cross-region database call, net 60 ms faster" is an explicit tradeoff backed by data, and you can defend it line by line.
Watch out
Don't optimize the median and call it done. If you only quote p50 in the room, expect the follow-up "what about p99?" Lead with the tail: name your queueing and placement risks before you're asked, and the follow-up becomes a chance to go deeper.