AWS Networking & Kubernetes at Scale

The VPC is the floor every Cursor service stands on. Get the address plan and the subnet topology wrong on day one and you pay for it in every migration, every peering and every region you add later.

A Software Engineer, Infrastructure at Cursor owns these network foundations for a product serving over 1M daily active users. The design-round bar is not reciting AWS terms. It is showing you can lay out a VPC that survives growth across regions and accounts without an IP renumbering project and that you know exactly where traffic is allowed to flow and where it is blocked.

Subnets, routing and AZ layoutthe skeleton

A VPC is one big CIDR block carved into subnets, one per Availability Zone per tier. A subnet is public if its route table sends 0.0.0.0/0 to an Internet Gateway and private if egress instead goes to a NAT Gateway. Production load balancers and bastions sit in public subnets. Everything that actually runs your workloads, EKS nodes included, sits in private subnets.

Security groups vs. NACLsstateful vs. stateless

Both filter traffic, but they live at different layers and behave differently. Security groups attach to an interface and are stateful: allow a connection out and the return packets come back automatically. NACLs sit at the subnet boundary and are stateless: you must write both directions, including ephemeral return ports or the reply is silently dropped.

Connectivity at scalepeering, Transit Gateway, PrivateLink

Two VPCs that need to talk can peer directly and that is fine for a handful. The trouble is that peering is non-transitive and the mesh grows as the square of the VPC count. Once you have many VPCs and accounts, a Transit Gateway becomes the hub everything routes through.

For reaching AWS services themselves (S3, ECR, Secrets Manager) from private subnets, use VPC endpoints. A gateway endpoint for S3 and DynamoDB or an interface endpoint for the rest keeps that traffic off the NAT path and off the public internet entirely. It cuts NAT cost and shrinks the attack surface in one move.

IP planning so growth doesn't corner youthe decision you can't easily undo

Pods on the EKS VPC CNI each consume a real VPC IP, so a busy cluster burns through address space fast. Plan CIDRs with that appetite in mind and reserve non-overlapping ranges per region and per account up front. Overlapping CIDRs are the thing that quietly blocks peering and Transit Gateway later and the fix is a renumbering project nobody wants.

IAM is where a small infra team either contains a compromise to one pod or hands an attacker the whole account. On a flat, talent-dense team, you own that boundary directly.

The design-round version of this question is rarely "what is an IAM role." It is "a credential leaks or a pod is compromised - how far does the damage spread and how did your design limit it?" Answer in terms of blast radiusHow much breaks if a change goes wrong; the scope of potential damage. and least privilege and ground it in how workloads actually get credentials on EKS.

Roles, policies and the two policy shapesthe vocabulary, used correctly

Least privilege means each principal carries the narrowest set of actions on the narrowest set of resources that lets it do its one job. Start from deny, add what breaks and scope with conditions (a specific resource ARN, a tag, a source VPC) rather than Resource: "*".

IRSA: pods get their own scoped rolethe EKS-specific answer

The naive way to give a pod AWS access is to attach a policy to the node's instance role. Then every pod on that node inherits it and one compromised container can act as all of them. IRSA (IAM Roles for Service Accounts) fixes this: the pod's Kubernetes service account is bound to an IAM role through an OIDCOpenID Connect. A modern standard that powers single sign-on, built on OAuth. provider, so each workload assumes its own scoped role and gets short-lived credentials.

Secrets and rotationno standing credentials in plaintext

• Store secrets in AWS Secrets Manager or SSM Parameter Store, encrypted with KMS and pull them at runtime - not at build time.
• Mount into pods via the Secrets Store CSI driver or External Secrets, scoped per workload via IRSA, so a pod only ever sees its own secrets.
• Rotate on a schedule and after any suspected exposure; prefer credentials that are short-lived by construction over ones you remember to rotate.

Containing blast radius by designboundaries, not just rules

The strongest containment is structural. Separate AWS accounts per environment mean a blown prod credential cannot touch staging and a misconfigured staging policy cannot reach prod data. Organization SCPs (Service Control Policies) sit above every role as a guardrail ceiling: even an over-broad role can't exceed what the SCP permits, so a mistake in one team's policy can't escalate past the account's hard limits.

You can't operate a cluster at 1M+ DAU scale by treating it as a black box that "runs containers." When pods go Pending or get OOM-killed at 3am, the person who knows the path from manifest to running process is the one who fixes it.

Control plane vs. data planewho decides vs. who runs

EKS splits cleanly. The control plane (API server, etcd, scheduler, controllers) is AWS-managed and decides what should run where. The data plane (your worker nodes running kubelet and a container runtime) is what you own and pay for and it does the actual running.

1. 1You apply a manifest. kubectl apply writes the desired Deployment to the API server, which persists it in etcd.
2. 2Controllers reconcile. The Deployment controller creates a ReplicaSet, which creates the Pod objects - still unscheduled.
3. 3The scheduler binds. It filters nodes by requests, taints and affinity, scores the survivors and binds each pod to a node.
4. 4Kubelet runs it. The node's kubelet sees the bound pod, pulls the image, asks the container runtime to start it and reports status back.
5. 5Probes gate traffic. Readiness must pass before the pod joins a Service's endpoints and starts taking requests.

Requests vs. limitsthe single most consequential knob

Requests are what the scheduler reserves; limits are the hard ceiling the kernel enforces. The gap between them decides scheduling density, eviction order and whether one noisy pod can starve its neighbors.

CNI and the IP-exhaustion trapwhere EKS networking bites

The AWS VPC CNI gives every pod a real VPC IP from the subnet's range, which makes pod networking native and fast. The cost is that a dense node with many pods consumes many subnet IPs and instances cap how many IPs they can attach by type. Run out of subnet IPs and new pods sit Pending with no obvious error in the app logs - the failure is in the network layer, not the workload.

• Size private subnets for peak pods-per-node times node count, then add headroom for rollouts that double pod count briefly.
• Watch for Pending pods with FailedCreatePodSandbox or IP-allocation events - that's CNI exhaustion, not a scheduling bug.
• Consider prefix delegation on the VPC CNI to pack more pod IPs per node or carve a secondary CIDR for pods specifically.

Health and safe rolloutsprobes and disruption budgets

A PodDisruptionBudget caps how many pods of a workload can be voluntarily down at once, so a node drain or cluster upgrade can't take a service below its minimum. Pair PDBs with a RollingUpdate strategy and correct readiness probes and you get zero-downtime deploys; skip them and a routine node rotation becomes an outage.

Stateful workloads are the exception on this platform, run via StatefulSets with stable network identities and per-pod PersistentVolumeClaims (EBS) that follow the pod's ordinal. They don't reschedule across AZs freely, because an EBS volume is AZ-bound, so storage topology becomes part of your HA design.

A service mesh is one of those tools that solves real problems and creates new ones. The senior signal isn't "I deployed Istio" - it's knowing the exact problems that justify the latency and operational tax and being willing to say no.

What a mesh actually buys youthe case for it

Istio vs. Linkerd vs. Ciliumand the sidecar tax

The classic mesh injects a sidecar proxy next to every pod, which intercepts all traffic. That's where the features come from and also where the cost comes from: a proxy per pod means extra CPU, memory and a couple of hops of latency on every request. The newer move is sidecarless - push mTLS and policy into the kernel with eBPF, so there's no per-pod proxy on the hot path.

How traffic enters the clusteringress and load balancers

External traffic reaches pods through ingress. An ingress controller watches Ingress (or Gateway API) resources and programs an AWS load balancer to match. On EKS the AWS Load Balancer Controller provisions an ALB for L7 routing or an NLB for L4 and the L4-vs-L7 choice shapes what you can do at the edge.

The honest counterpointwhen NOT to run a mesh

A mesh adds a control plane to operate, upgrade and debug; it adds latency; and it makes failures harder to reason about because the proxy is now in the path of every request. For a small number of services, you can get mTLS from the edge plus app-level TLS, retries from a shared client library and traffic splitting from the ingress alone - without the mesh tax.

Autoscaling for 1M+ DAUs is a control-systems problem. Scale on the wrong signal and you over-provision a fortune; tune it badly and you get oscillation that hurts more than the load ever would.

Three layers, three jobspods, sizing and nodes

Scaling happens at distinct layers and conflating them is a common interview stumble. One scales pod count, one scales each pod's resources and one scales the nodes underneath.

Scale on SLO-relevant signalsnot just CPU

CPU is a proxy and often a bad one. A request queue can be backing up while CPU sits at 40%, because the work is I/O-bound or waiting on a model backend. Scale on the thing your users actually feel: p99 latency, queue depth or in-flight request concurrency.

Surviving spiky, bursty trafficscale-up latency is the enemy

The hard truth of bursty load is that adding a node isn't instant. A new EC2 instance must boot, join the cluster, pull images and pass readiness, which can take minutes - and a traffic spike doesn't wait. So you buy time with headroom and warm capacity rather than reacting from zero.

• Keep a headroom buffer (over-provisioning / pause pods) so a spike lands on already-running capacity while new nodes boot.
• Use warm pools or fast-launching node types so scale-up latency is seconds, not minutes.
• Pre-pull large images to nodes so image fetch isn't on the critical path during a spike.
• Set HPA min replicas above your true floor for spiky services, so you never scale up from a cold, single-pod state.

Avoiding thrashthe feedback-loop failure

An autoscaler that reacts too fast oscillates: it scales up on a brief spike, the metric drops, it scales down, the load returns and it scales up again - churning pods and nodes while serving worse than a steady fleet would. Stabilization windows and asymmetric up/down behavior tame it.