Edge, Security & Multi-Region Deployment

Before a request ever touches your EKS cluster, it crosses four or five edge layers. In a Cursor system-design round, the candidates who lose are the ones who jump straight to pods and ingress and skip everything that happens at the planet's surface.

Cursor serves 1M+ daily active users from a desktop app that talks to backends for indexing, retrieval and model inference. Most of that traffic is latency-sensitive and a lot of it is expensive to serve. The edge is where you decide what gets cached, what gets blocked and which region a user lands in - before any of it costs you compute.

Walk the path a request takes, in order. Naming the layers buys you credibility before you've designed anything.

1. 1DNS resolution. The client resolves your hostname. With Anycast and geo-aware DNS, the answer it gets already steers toward a nearby edge POP.
2. 2Anycast / edge POP. The same IP is announced from many locations; BGP routes the user to the closest one. TCP and TLS handshakes terminate here, near the user, not in us-east-1.
3. 3CDN cache check. If the response is cacheable and warm, the edge answers immediately and origin never sees the request.
4. 4WAF + edge logic. Rules inspect the request for known attack classes; rate limiting and bot checks run here so junk traffic is shed early.
5. 5Origin / regional load balancer. Only the surviving, uncacheable requests cross to a region's ALB and into the mesh.

What is and isn't cacheable for a developer toolthe CDN is only as useful as your cache hit ratio

A CDN earns its keep on static, shared, immutable bytes. The trap for an AI dev tool is that the high-value traffic is the least cacheable: a user's prompt, their repo context and a streamed model completion are all personalized and one-shot. Be specific about the split.

WAF, TLS termination and mTLS to originwhere encryption ends and where trust begins

TLS should terminate at the edge for handshake latency, then a fresh, mutually authenticated TLS session should carry the request to origin. Terminating once at the edge and sending plaintext over your backbone is the cheap mistake that fails a security review.

Rate limiting is where infra meets unit economics. Every model call has a hard dollar cost, so a runaway script or a stolen API key isn't just a reliability problem - it's a bill.

The interviewer wants to see that you can pick an algorithm for a reason, run it across many edge nodes without a hot key meltdown and degrade kindly when a real user brushes the limit. Start with the algorithms.

For Cursor, token bucket fits the human pattern: a developer fires several agent requests in a flurry, then goes quiet to read the diff. You want to allow the flurry and cap the sustained rate, which is exactly the burst + refill shape.

Distributed rate limiting and hot keysthe part that breaks at scale

One counter in memory is easy. The hard version is hundreds of edge nodes that must agree on whether a user is over their limit, in single-digit milliseconds, without a central counter becoming a bottleneck. Two real problems show up.

Choose the right limiting dimension

Abuse, DDoS and protecting expensive backendsthe inference path is the crown jewel

DDoS volumetrics get absorbed by Anycast and scrubbing at the edge. The subtler threat is abuse that looks legitimate: a credential-stuffed account or a leaked token driving thousands of real, expensive model calls. Defend the wallet, not just the wire.

• Tier limits by cost: a cheap autocomplete endpoint gets a generous bucket; a long-context agent run gets a tight one tied to plan.
• Add an anomaly signal on spend velocity per account, not just request count, so a sudden cost spike trips a challenge.
• Use challenge flows (a managed bot check) for suspicious clients before you outright block, to avoid false-positive lockouts.
• Return honest 429s with a Retry-After hint so well-behaved clients back off instead of retry-storming you.

HTTP/1.1 429 Too Many Requests
Retry-After: 8
RateLimit-Limit: 60
RateLimit-Remaining: 0
RateLimit-Reset: 8
Content-Type: application/json

{
  "error": "rate_limited",
  "scope": "per_account",
  "retry_after_seconds": 8
}

Going multi-region is a tax you pay for two things: lower latency for distant users and survival when a region fails. Pick the topology that buys the resilience you actually need, not the most impressive diagram.

The first fork is active-active versus active-passive. The honest framing is cost and operational complexity against how fast you must recover. Lay it out before you commit.

Data locality and replicationthe part that decides whether multi-region even works

Stateless services are easy to spread; data is the hard constraint. A cross-region database call adds 60–150 ms of speed-of-light tax every time, so a chatty path that hops the Atlantic per request will be slow no matter how good your edge is. The design goal is to keep each request's data reads inside its own region.

• Read-local, write-where-it-makes-sense: serve reads from a same-region replica; route writes to the owning region or use a database built for multi-region writes if conflicts are rare.
• Pin a user's session to a home region so their hot data stays close and you avoid replication-lag surprises mid-session.
• Keep large/cold artifacts (indexes, blobs) in regional object storage replicated lazily, not in the synchronous request path.
• Name your consistency choice out loud: strong within region, eventual across regions and which user-visible operations can tolerate which.

Failover and traffic steeringtested, not theoretical

Steering lives in two places: DNS / global load balancer health checks decide which region is even an option and the edge picks among the healthy ones. The credibility line is whether failover is tested.

1. 1Health checks at the region level probe a real dependency path, not just a port being open - a region that can't reach its database is unhealthy even if its ALB answers.
2. 2Global LB / geo-DNS removes an unhealthy region from rotation and steers users to the next-nearest healthy one.
3. 3Drain and rebalance so the surviving regions don't fall over from the displaced load - capacity-plan for N-1.
4. 4Game-day it. Kill a region on purpose in a drill; an untested failover is a hope, not a control.

Multi-cloud: when the operational tax is worth ita second provider is a big commitment

Multi-region on one cloud handles most resilience needs. A second cloud provider doubles your IAM models, networking primitives and on-call surface and you lose the deep integrations that make one cloud productive. Reach for it only with a concrete driver.

Cursor is latency-obsessed by design and it shows in the interview. "Make it fast" is a junior answer. "Here's my millisecond budget and where I'd spend or save" is the one that lands.

Break the round trip into named components, give each a budget and treat the sum as a contract. When you blow the budget, you'll know exactly which segment to attack instead of guessing.

Inference is its own world: a streamed completion's time-to-first-token matters more than total time, because the user sees motion immediately. Budget the first token tightly and let the stream fill in.

p50 versus p99: design for the tailaverages lie at scale

At 1M+ DAU, the p99 is not a rare edge - it's tens of thousands of requests an hour and it's disproportionately your most engaged users hitting it repeatedly. A great median with an ugly tail still feels broken.

Techniques to buy milliseconds backwhere the savings actually are

• Connection reuse and HTTP/2 or HTTP/3: kill repeated handshakes - often the cheapest big win for chatty clients.
• Regional pinning: put the user, their data and their compute in one region so the network segment stays small.
• Cache the cacheable at the edge so those requests never pay backend or network cost at all.
• Shed work to the edge: do auth, rate-limit and routing decisions near the user and short-circuit early.
• Hedged requests for the tail: on a slow outlier, fire a second request and take the first to return - spend a little compute to cut p99.

Everything in this module - the edge, the limits, the regions, the budgets - should hang off one number: the SLO. It turns "is this reliable enough?" from a vibe into a decision you can defend.

An SLI is what you measure, an SLO is the target you commit to and the error budget is the allowed shortfall. The budget is the lever: spend it on shipping when you're healthy, freeze and stabilize when you've burned it.

The cultural payoff fits Cursor's bias to ship: a healthy error budget is permission to move fast and take risks. A burned budget is the signal to slow down and harden. The number, not opinion, arbitrates the ship-versus-stabilize fight.

Golden signals and alerting on symptomsalert on what users feel

The discipline behind "alert on symptoms, not causes" is that one user-facing symptom can have many causes and you'll burn out an on-call rotation paging on every cause that might one day matter. Page when the SLO is at risk; investigate the cause once you're already awake.

Graceful degradation and circuit breakersa partial outage shouldn't be a total one

When a dependency or a region fails, the goal is to lose a feature, not the product. Circuit breakers stop you from hammering a sick dependency and dragging healthy paths down with it.

Observability across regionsyou can't debug what you can't trace

A request that crosses edge → mesh → service across regions is invisible without distributed tracing. Propagate a trace context from the edge down and correlate it with metrics and logs so one slow request tells you which hop ate the time.

1. 1Inject a trace/request ID at the edge and carry it through every hop and every log line.
2. 2Trace the path edge → ingress → mesh sidecar → service → datastore, with spans you can read as a waterfall.
3. 3Tag spans by region and node so a region-localized problem stands out instead of averaging away.
4. 4Correlate the three pillars - jump from a latency metric to the exemplar trace to the logs for that exact request.