1 min lesson
Techniques to buy milliseconds back
Walk through the important items in "Techniques to buy milliseconds back" and give the practical point of each.
Step 1 of 2
Techniques to buy milliseconds backwhere the savings actually are
- Connection reuse and HTTP/2 or HTTP/3: kill repeated handshakes - often the cheapest big win for chatty clients.
- Regional pinning: put the user, their data and their compute in one region so the network segment stays small.
- Cache the cacheable at the edge so those requests never pay backend or network cost at all.
- Shed work to the edge: do auth, rate-limit and routing decisions near the user and short-circuit early.
- Hedged requests for the tail: on a slow outlier, fire a second request and take the first to return - spend a little compute to cut p99.
Budget, then spend
The discipline is to set the budget first, instrument each segment and then make spend/save decisions against data. "We added 30 ms of edge auth to remove a 90 ms cross-region database call, net 60 ms faster" is the kind of explicit tradeoff that reads as a Cursor-grade infra engineer.
Watch out
Don't optimize the median and call it done. If you only quote p50 in the room, expect the follow-up "what about p99?" Lead with the tail: name your queueing and placement risks before you're asked and you've shown the instinct the team actually screens for.