1 min lesson
Distributed rate limiting and hot keys
For each case in "Distributed rate limiting and hot keys", name the signal and the response you would use.
Step 1 of 2
Distributed rate limiting and hot keysthe part that breaks at scale
One counter in memory is easy. The hard version is hundreds of edge nodes that must agree on whether a user is over their limit, in single-digit milliseconds, without a central counter becoming a bottleneck. Two real problems show up.
A single heavy user (or a viral repo's CI) hammers one counter shard.
Mitigate with local token buckets per node plus periodic reconciliation to a shared store.
Accept small over-admission as the price of not centralizing every decision.
A low-latency store (Redis-style) holds the authoritative count.
Use atomic ops so concurrent nodes don't double-count.
Async replication across regions; per-region local enforcement to avoid a cross-region round trip on every request.
Learn more
Full explanation
Choose the right limiting dimension
Choose the right limiting dimension
- Per API key
- Best for programmatic access and billing; the key is the identity and the cost center.
- Per user / account
- Maps limits to entitlements and plan tier; survives IP changes.
- Per IP
- Last line against anonymous floods; weak alone (NAT, shared offices, CGNAT) so use as a coarse net, not the primary control.
Real systems layer all three: per-key for cost, per-account for fairness, per-IP for the anonymous edge.