2 min lesson
GPU economics fundamentals
In "GPU economics fundamentals", choose the best answer to "Which statement best captures why idle GPUs are the biggest waste lever?" Then say what led you there.
Step 1 of 2
A GPU costs roughly the same whether it runs at 90% or 9%. That one fact reshapes how you think about every dollar of Cursor's inference bill.
Inference is the dominant COGS line at a company serving developers at very large scale and the asset behind that line is a fleet of accelerators that are expensive to acquire, hungry for power and impossible to pause. Unlike a CPU box you can scale to zero, a reserved H100 bills the same per hour at idle as under full load. So the real cost question is never just "how many GPUs" - it is "how busy are the ones we already pay for."
When a systems-literacy interviewer asks what drives GPU cost, they want the cost structure, not a list of part numbers. There are four levers and idle time is the one that quietly wastes the most.
- Acquisition / amortization
- The card itself, amortized over its useful life - fixed whether you use it or not
- Power and cooling
- High sustained draw per accelerator; a real line in any owned or colo footprint
- Idle time
- Hours you pay for and get nothing back - the single biggest waste lever
- Acquisition lead time
- Not a dollar directly, but supply scarcity forces commitments months ahead of demand
Idle time is the lever you manage; the other three are mostly given to you.