IaC, Cost Engineering & Platform Unification
Reproducible infra, cost attribution and one opinionated compute platform
Infrastructure-as-code, done right
After this you can build reproducible infra and roll changes out safely.
The job description says it plainly: codify everything as reproducible infrastructure-as-code. In the loop, that line becomes a question - could a new hire rebuild your production network from an empty AWS account using only what's in git?
At Cursor's scale, IaCInfrastructure as Code. Managing servers and cloud resources through version-controlled config files (e.g. Terraform). is the substrate every other answer in this module rests on. You can't attribute cost to a resource you provisioned by hand and you can't unify a platform whose clusters were each clicked together in the console. So the interviewer is rarely asking whether you've used Terraform. They want to see whether your IaC is the source of truth or just a record of what someone hoped the infrastructure looked like.
Module design and environment parityDRY without the magic
A good module is a small, versioned contract: a tight input surface, sane defaults and outputs other modules can consume. The trap is over-abstraction - a module so parameterized that reading it tells you nothing about what it builds. Aim for composition over a single mega-module and let environments differ by variable values, not by forked code.
module "vpc" {
source = "git::https://internal/modules//vpc?ref=v3.2.1"
name = var.env # dev | staging | prod
cidr = var.vpc_cidr
azs = var.azs # prod spans 3 AZs, dev spans 2
nat = var.env == "prod" ? "per-az" : "single"
tags = local.required_tags
}
# envs/prod.tfvars
env = "prod"
vpc_cidr = "10.0.0.0/16"
azs = ["us-east-1a", "us-east-1b", "us-east-1c"]?ref=v3.2.1 is doing real work. An unpinned source means a teammate's apply can silently pull a newer module and reshape prod. Pin module versions and provider versions and treat a version bump as a reviewed change like any other.
State, locking and driftthe part that bites in production
Remote state with locking is non-negotiable on a team. Two engineers applying against the same unlocked state can corrupt it and a corrupted state file is a bad afternoon. The deeper signal is how you treat drift - the gap between what code says and what the cloud actually runs.
- Concept
- Remote state
- What it is
- The recorded mapping from code to real resources
- How a strong answer handles it
- S3 backend with versioning; never local, never in git
- Concept
- State locking
- What it is
- Mutual exclusion so two applies can't race
- How a strong answer handles it
- DynamoDB lock table (or backend-native locking); CI holds the lock
- Concept
- Drift
- What it is
- Reality diverging from code (console edits, manual fixes)
- How a strong answer handles it
- Scheduled
planin CI; a non-empty plan on an unchanged branch is an alert
- Concept
- Reconciliation
- What it is
- Closing the drift gap
- How a strong answer handles it
- Import or codify the change, never re-click; ask why the bypass happened
| Concept | What it is | How a strong answer handles it |
|---|---|---|
| Remote state | The recorded mapping from code to real resources | S3 backend with versioning; never local, never in git |
| State locking | Mutual exclusion so two applies can't race | DynamoDB lock table (or backend-native locking); CI holds the lock |
| Drift | Reality diverging from code (console edits, manual fixes) | Scheduled plan in CI; a non-empty plan on an unchanged branch is an alert |
| Reconciliation | Closing the drift gap | Import or codify the change, never re-click; ask why the bypass happened |
Drift detection turns "someone changed prod" from a mystery into a ticket.
Treating drift as a shrug is the tell of someone who's run IaCInfrastructure as Code. Managing servers and cloud resources through version-controlled config files (e.g. Terraform). as decoration. A console hotfix during an incident is sometimes correct - but if it never gets reconciled back into code, your next clean apply will quietly revert it and re-break prod. Drift is an incident with a delay timer.
Safe rollout and blast radiusplans are reviewed code
- 1Plan in CI on the PR. Post the
planoutput as a comment so reviewers see exactly what will change before approval. A diff nobody read is a diff nobody owns. - 2Apply from CI, never a laptop. Humans propose, the pipeline applies. This gives you an audit trail and one set of credentials to govern.
- 3Stage the apply. Roll dev, then staging, then prod from the same code. The change earns trust at each ring before it touches the layer 1M+ users sit on.
- 4Bound the blast radiusHow much breaks if a change goes wrong; the scope of potential damage.. Split state by domain (network, data, compute) so a bad apply to one can't take the others down with it.
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
Humans propose; the pipeline applies - and the change earns trust ring by ring.
When asked how you ship an infra change, anchor on blast radiusHow much breaks if a change goes wrong; the scope of potential damage. before you mention any tool. Say: "I'd split state so a network change can't break the data plane, plan on the PR and stage the apply through dev and staging first." That sequencing is the senior signal - the tradeoff between speed and containment is exactly what an infra interviewer is listening for.
Takeaway. IaCInfrastructure as Code. Managing servers and cloud resources through version-controlled config files (e.g. Terraform). earns its name only when it's the source of truth: pinned modules, locked remote state, drift treated as an incident and applies that stage through rings from a pipeline - not a laptop.
Self-check
QA scheduled terraform plan runs nightly against the prod branch with no code changes and one morning it shows a non-empty plan. What does that mean and what do you do?
Cloud cost attribution systems
After this you can make spend legible per team, service and feature.
Cost at Cursor isn't a finance afterthought. The JD calls cost a first-order business problem, because a product serving 1M+ DAUs with GPU-heavy inference can burn money in places nobody's looking. The first job is making the spend legible.
An undifferentiated cloud bill is a number you can't act on. Attribution turns "we spent $1.2M last month" into "the agent inference service spent $640K, retrieval indexing spent $180K and a forgotten staging cluster spent $40K." The mechanism that makes this possible is boring and ruthless: every resource carries tags that say who owns it and why it exists.
Tag governance through CI/CDattribution is enforced, not requested
Asking teams to tag nicely produces an 80%-untagged bill. The only durable answer is enforcement at provision time, where the cost actually originates. Tagging lives in the same IaCInfrastructure as Code. Managing servers and cloud resources through version-controlled config files (e.g. Terraform). pipeline from the last section, which is why the two halves of this role are joined at the hip.
- team
- Owning team - the chargeback target (e.g. infra, agent, web)
- service
- Logical service the resource belongs to (e.g. inference-gateway)
- env
- dev | staging | prod - separates real spend from experiments
- cost-center
- Finance grouping for showback rollups
- managed-by
- terraform | manual - anything manual is a drift and a cost mystery
Five tags, mandatory at apply time. Untagged resources fail the plan, not the audit.
# OPA/Conftest-style guard run in CI before apply
deny[msg] {
resource := input.resource_changes[_]
required := {"team", "service", "env", "cost-center"}
missing := required - {k | resource.change.after.tags[k]}
count(missing) > 0
msg := sprintf("%s missing tags: %v", [resource.address, missing])
}CI tag enforcement only covers what flows through IaCInfrastructure as Code. Managing servers and cloud resources through version-controlled config files (e.g. Terraform).. Anything created in the console or by an autoscaler escapes it. Pair the gate with a daily sweep that flags untagged or managed-by=manual resources, so the legibility you built at provision time doesn't quietly erode.
Showback, chargeback and unit economicsfrom totals to behavior change
- Model
- Showback
- What it does
- Shows each team its spend; no money moves
- When it's the right call
- Early - build awareness and trust before you bill anyone
- Model
- Chargeback
- What it does
- Bills the spend to the team's budget
- When it's the right call
- When teams have budgets and the data is trusted enough to defend
- Model
- Unit economics
- What it does
- Cost per meaningful unit (per DAU, per 1K completions)
- When it's the right call
- When you need to reason about whether a feature is even affordable
| Model | What it does | When it's the right call |
|---|---|---|
| Showback | Shows each team its spend; no money moves | Early - build awareness and trust before you bill anyone |
| Chargeback | Bills the spend to the team's budget | When teams have budgets and the data is trusted enough to defend |
| Unit economics | Cost per meaningful unit (per DAU, per 1K completions) | When you need to reason about whether a feature is even affordable |
Most orgs earn the right to chargeback by running clean showback first.
Unit economics is the framing that matters most for Cursor. A raw total tells you the bill grew. Cost per 1,000 completions tells you whether it grew because you have more users (good) or because the inference path got more wasteful (a problem to chase). That ratio is the language a cost-aware infra engineer speaks.
Splitting shared coststhe honest-hard part
Some spend won't tag cleanly: a shared EKS control plane, a multi-tenant cluster, the NAT gateway every service egresses through. Pretending these are free hides real money. The move is to pick a defensible allocation key and be transparent that it's an approximation, not a measurement.
Allocate by each team's requested CPU/memory (or actual usage).
Kubecost or OpenCost reads this straight from pod requests.
A fixed "platform tax" spread proportionally to consumption.
Name it explicitly so teams see the cost of the paved road.
Attribute by source service from VPC flow logs where you can.
What you can't split, pool - and make the pool visible, not hidden.
Split by log volume per service or namespace.
Often a surprise line item worth attributing on its own.
Takeaway. Attribution is enforced at provision time - required tags gated in CI - then turned into unit economics (cost per 1K completions, not just totals) with shared costs split by an honest, named allocation key.
Self-check
Identifying and removing waste
After this you can find the dollars and cut them without hurting reliability.
Attribution tells you where the money goes. This section is about getting it back. Waste in a fast-growing infra org is rarely one villain - it's a hundred small leaks that a clean attribution layer finally makes visible.
The discipline that separates a strong answer from a hand-wave is measurement. "We right-sized the cluster" is a story. "We cut node count 30% by dropping p95-padded requests and p99 latency held" is engineering. Quantify before and after, every time.
Right-sizing requests, limits and nodesthe largest reclaimable pool
On Kubernetes, the most common waste is requests set far above real usage. The scheduler packs nodes by requests, so inflated requests mean you pay for nodes running near-empty. Right-sizing requests to observed usage (with headroom for spikes) lets the bin-packer do its job.
- Set requests near observed p95 usage with deliberate headroom; let VPA recommend, but apply with judgment, not blindly.
- Keep CPU limits loose or unset so latency-sensitive pods can burst; keep memory limits tight, since memory overcommit gets pods OOM-killed.
- Let Karpenter or Cluster Autoscaler consolidate underused nodes and pick instance types from the pod shapes, not habit.
- Reclaim the obvious dead weight first: zombie staging clusters, orphaned EBS volumes, idle load balancers and unattached elastic IPs.
Right-sizing requests down increases bin-packing density, which means a node failure now evicts more pods at once. Cheaper and denser is also more fragile. Name that tradeoff out loud rather than presenting density as a free win - it's the exact cost-vs-reliability judgment the next section drills.
Matching commitment to workload shapeSpot, Savings Plans, Reserved
- Pricing model
- On-demand
- Best fit
- Spiky, unpredictable or short-lived workloads
- The catch
- Most expensive per hour; fine as the flexible top layer
- Pricing model
- Spot
- Best fit
- Interruptible, stateless, replayable (batch, CI, some inference)
- The catch
- Can be reclaimed with ~2 min notice; needs graceful drain
- Pricing model
- Savings Plans
- Best fit
- Steady baseline compute you'll run regardless of instance type
- The catch
- 1- or 3-year commit; you pay even if usage drops
- Pricing model
- Reserved
- Best fit
- Stable, known instance families (often databases)
- The catch
- Least flexible; stranded if the workload shape changes
| Pricing model | Best fit | The catch |
|---|---|---|
| On-demand | Spiky, unpredictable or short-lived workloads | Most expensive per hour; fine as the flexible top layer |
| Spot | Interruptible, stateless, replayable (batch, CI, some inference) | Can be reclaimed with ~2 min notice; needs graceful drain |
| Savings Plans | Steady baseline compute you'll run regardless of instance type | 1- or 3-year commit; you pay even if usage drops |
| Reserved | Stable, known instance families (often databases) | Least flexible; stranded if the workload shape changes |
Match the commitment to the workload's shape: commit the floor, flex the peak.
The mental model is a stack. Cover your steady 24/7 baseline with Savings Plans or Reserved, ride Spot for anything interruptible and let on-demand absorb the spiky top. Over-committing strands money on a 3-year plan; under-committing leaves savings on the table. The shape of the workload decides, not a blanket policy.
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
Layer pricing models by how predictable the workload underneath them is.
Compute hot spots and egresswhere the big numbers hide
The most expensive idle in an AI shop, by far.
Bin-pack inference, scale to zero off-peak and alert on low utilization - an idle H100 is a five-figure monthly leak.
Preview/staging stacks that outlive their PR.
TTL labels plus a reaper that tears them down automatically.
Chatty services split across AZs pay per-GB both ways.
Use topology-aware routing to keep traffic in-zone where safe.
Cacheable responses hitting origin burn compute and egress.
Push a CDN in front; measure cache hit ratio before and after.
Every waste claim needs a before/after number. "Added a CDN, cache hit ratio went 0% to 82%, origin requests dropped 5x, egress fell $18K/mo" is a defensible win. "Added a CDN, should help" is a guess wearing a confident face - and an interviewer will ask for the number.
Takeaway. Find waste through attribution, attack the biggest pools first (oversized requests, idle GPUs, zombie envs, cross-AZ egress), commit the baseline while flexing the peak and quantify before/after - never guess.
Self-check
QAn interviewer asks you to cut the EKS bill 25%. You're tempted to slash all pod requests to their bare minimum. What's the danger and what's a more defensible move?
The cost-vs-reliability tradeoff
After this you can make and defend explicit tradeoffs the JD calls for.
This is the section the JD is really about. "Make smart cost-versus-reliability tradeoffs" is in the responsibilities and "cost/reliability judgment - making explicit, defensible tradeoffs rather than gold-plating" is in the behavioral themes. The interview probes this directly and a fuzzy answer is disqualifying.
The frame that separates senior from junior is the question you ask. A junior reaches for cheapest or most reliable. A senior asks what reliability they're buying per dollar, then decides whether that reliability is worth its price for this specific system.
Don't ask "how cheap can this be" or "how reliable can we make it." Ask: "what reliability does this dollar buy and does this system need it?" A third nine of availability on an internal dashboard and a third nine on the inference path are wildly different purchases at a similar price.
Error budgets are the dialspending reliability you've already promised
An error budget turns reliability from a vibe into a number you can spend. If the SLO is 99.9% and you're consistently hitting 99.99%, you are over-delivering - and over-delivery often means over-paying for redundancy nobody asked for. That gap is a budget you can spend on cost cuts or velocity.
- Burning budget fast
- Buy reliability: add redundancy, slow risky changes, invest in resilience
- Hitting SLO with margin
- Hold - you're priced about right; don't gold-plate further
- Consistently far above SLO
- Spend the surplus: remove redundancy that isn't paying for itself, take more velocity
- No SLO at all
- You can't reason about the tradeoff yet; define the SLI/SLO first
The error budget tells you which direction to move on the cost/reliability dial.
Removing redundancy feels dangerous, so make it evidence-led. If a service has held three nines for a year against a 99.9% SLO, dropping from a hot standby in a third region to a warm one is spending surplus budget, not gambling. State it that way and the move stops sounding reckless.
Where Cursor spends vs. savesrole-specific judgment
Latency on the hot path - Cursor is latency-obsessed and users feel every extra 100ms in completions.
Abuse and DDoS defense at the edge - an outage or abuse spike is a first-order business risk at 1M+ DAU.
Multi-region resilience for user-facing inference where a regional failure is visible.
Idle compute and over-provisioned headroom that no SLO requires.
Gold-plated redundancy on internal/back-office systems users never touch.
Three-region hot standbys for services a warm standby would cover.
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
Same dimensions, opposite calls - the difference is how much reliability the dollar buys here.
"I wouldn't make that call on cost alone. Inference latency is core to how Cursor feels, so I'd protect the hot path even at a premium. But that internal batch job has run for a year inside its 99.9% SLO with budget to spare - I'd move it from three regions to two, bank the savings and watch the burn rate. If the budget starts burning, we add it back."
When they hand you an open tradeoff, never answer with a single number. Ask one clarifying question - what's the SLO, who depends on this - then commit to a decision with its reasoning. Truth-seeking plus decisiveness is the exact pairing Cursor screens for; refusing to decide reads as worse than deciding wrong.
Takeaway. Frame every decision as reliability bought per dollar, use the error budget as the dial - spend surplus, buy more when it's burning - and protect Cursor's hot path while cutting redundancy no SLO requires.
Self-check
Unifying the compute platform
After this you can plan a migration to one opinionated, paved-road platform.
The JD's last responsibility is the most ambitious: unify the compute platform so every team gets consistent, reliable deployments out of the box. This is platform-as-product work and it's where a strong candidate shows they think about internal customers, not just clusters.
The problem it solves is sprawl. When every team builds its own deploy pipeline, you get N bespoke setups, N ways to fail and an infra team stretched across all of them. A single opinionated platform trades some per-team flexibility for reliability-by-default - and on a small, flat team, that impact is the whole point.
Paved roads and golden pathsthe well-lit default, not a wall
A paved road is the supported, well-lit way to ship: sensible defaults, guardrails baked in, self-service instead of tickets. The art is that it's a default, not a prison. Teams stay on it because it's genuinely the easiest path and the rare team that steps off does so knowingly and owns what they took on.
- Deploy
- GitOps via Argo - merge to deploy, no bespoke pipeline to maintain
- Observability
- Metrics, logs, traces and golden-signal dashboards wired in for free
- Scaling
- Sane HPA defaults and autoscaling on the right signals out of the box
- Security
- Least-privilege IAM via IRSA, mTLS from the mesh, secrets handled for you
- Cost
- Required tags and the attribution layer applied automatically
Every default here is a decision a team no longer has to get right alone.
The platform succeeds when a team can go from new service to production without filing a ticket or waiting on infra. Every manual handoff you remove is impact on a small team - and it's the difference between a platform and a bottleneck wearing a platform's name.
Migration without freezing deliverystrangler, not big-bang
Nobody lets you stop the world to re-platform a product at 1M+ DAU. The strangler pattern is the answer: stand the new platform up alongside the old, route new and willing workloads onto it and let the legacy footprint shrink until it's gone. Teams keep shipping the entire time.
- 1Build the road for one real workload. Onboard a friendly team's actual service end to end. A platform validated on a toy never survives the first real edge case.
- 2Make it the default for the new. Every net-new service starts on the paved road. You stop growing the legacy sprawl before you try to shrink it.
- 3Migrate by carrot, then incentive. Lead with teams that feel the pain; let "free observability and no pipeline to babysit" sell it. Mandate only once the road is clearly better.
- 4Strangle the legacy. Move existing services in waves, deprecate the old paths and delete bespoke per-team infra as each wave clears.
- 5Keep teams shipping throughout. Both paths stay live during the move; no team's delivery stalls so the migration can look tidy.
A mandate before the road is genuinely better breeds shadow infra - teams quietly building around your platform because it slows them down. Win adoption by being the easiest path first. Force it too early and you've created the exact sprawl you set out to kill, now with worse morale.
Measuring successadoption is the scoreboard
- Metric
- Adoption rate
- What it tells you
- Share of services on the paved road
- Healthy direction
- Rising toward near-total; plateaus flag friction
- Metric
- Time-to-first-deploy
- What it tells you
- New service to production for a fresh team
- Healthy direction
- Falling from days to under an hour
- Metric
- Bespoke infra count
- What it tells you
- Per-team one-off pipelines and clusters
- Healthy direction
- Falling toward zero as waves complete
- Metric
- Change failure rate
- What it tells you
- Deploys that cause an incident
- Healthy direction
- Falling - the whole point of reliability-by-default
| Metric | What it tells you | Healthy direction |
|---|---|---|
| Adoption rate | Share of services on the paved road | Rising toward near-total; plateaus flag friction |
| Time-to-first-deploy | New service to production for a fresh team | Falling from days to under an hour |
| Bespoke infra count | Per-team one-off pipelines and clusters | Falling toward zero as waves complete |
| Change failure rate | Deploys that cause an incident | Falling - the whole point of reliability-by-default |
If adoption is voluntary and still climbing, the road is genuinely the best path.
If they ask you to design the platform migration, lead with adoption strategy before architecture. Say you'd win the first team with a carrot, default all new services onto the road and strangle the legacy in waves while everyone keeps shipping. Showing you'd measure adoption and time-to-first-deploy proves you treat platform as a product with customers - exactly the framing this role rewards.
Takeaway. Unify with a paved road - sensible defaults, guardrails, self-service over tickets - adopted by carrot before mandate, migrated via the strangler pattern so teams never stop shipping and scored on adoption and time-to-first-deploy.
Self-check
QLeadership wants you to unify everyone onto one platform fast and proposes mandating it next quarter. Why might a hard mandate backfire and what would you do instead?