2 min lesson
Model routing
Rebuild the sequence in "Model routing" from memory, ending with the check that proves the outcome.
Step 1 of 3
Model routingsend the easy work to the cheap model
Not every request deserves the biggest model. Model routing sends simple completions to a small, cheap, fast model and reserves the large model for genuinely hard queries. Done well it cuts cost and latency at the same time, because the small model is both cheaper to run and quicker to respond. The risk is misrouting: a hard query sent to the small model returns a weak answer, so the router's accuracy is itself a quality lever.
Interactive diagram. Step through it with the Next and Previous controls below, or Tab to a region to read its detail.
Routing is a loop, not a switch - the measure step is the gate that catches misrouting before it ships.
Learn more
Full explanation
Full explanation
When asked "how would you lower cost per inference," do not name one lever. Walk the four together: route simple queries to a smaller model, batch where the latency budget allows, raise utilization on the committed fleet and verify the change against p95/p99 so you have not traded a cost win for a latency regression. Closing on the verification step is what separates a TPM from someone reciting tricks.