1 min lesson
GPU economics and provider model
Use "GPU cost is mostly about keeping expensive silicon busy" to describe the practical rule.
Step 1 of 3
Everything in the previous four sections eventually becomes a number on a bill or a capacity plan. The routing engineer who can do this math is the one who earns the gray-area decisions.
GPU cost is mostly about keeping expensive silicon busy. The levers that move it are the ones you already met: paged KV, continuous batching and prefix-cache hits.
Learn more
Full explanation
From Silicon to the Dollar
Interactive diagram. Tab through its regions; each focused region shows its detail in the panel below.
diagram: stack
Each layer rests on the one below: cost is set at the bottom, then earned back by how well the layers above pack it.
The cost vocabulary you'll be tested on
- Per-token pricing
- Separate input (prefill) and output (decode) rates; output is usually pricier
- Rate limits
- RPM (requests/min) and TPM (tokens/min) caps per key/account