1 min lesson
Caching, batching, prompt reuse
Rebuild the parts of "Caching, batching, prompt reuse", then say why each one matters.
Step 1 of 3
Caching, batching, prompt reusecut cost and latency together
- Prompt / prefix caching
- Reuse the cached prefix (rules, system prompt, stable context) so repeated runs only pay for the new tokens.
- Result caching
- Identical input + revision returns the prior result instead of re-calling the model.
- Batching
- Group independent calls (e.g. one per file in a refactor) to amortize overhead and raise throughput.
Learn more
Full explanation
Token budgeting
Token budgetingmeasure before you ship
You can’t defend a cost you never measured. Instrument tokens per task during the build, then set a ceiling so a pathological input can’t quietly 10x the bill in production.
Log input and output tokens per task and watch the p50 and p95, not just the average.
Set a per-task ceiling that aborts or downgrades the model before spend runs away, the same way you cap steps in the loop.
Learn more
Full explanation
Optimize for the right tokens, not the fewest
Optimize for the right tokens, not the fewesttrue efficiency = efficiency × quality
Cheapest-per-token is the wrong target. A token-cheap model that needs 100 re-prompts isn’t efficient, and a model that one-shots a trivial feature while burning a billion tokens isn’t either. Tokens map straight to dollars - each model has distinct input/output/cached rates - so small inefficiencies compound at scale, but the goal is good hygiene that happens to produce cost efficiency and a better experience. Don’t penny-pinch the per-prompt cost; spend the right tokens on the right step.
The flagship loop: use your smartest thinking modelA reasoning model (shown with a brain icon in Cursor's picker) that spends extra compute before answering; reach for it on complex, nuanced work and a standard model for fast, simple tasks. Press Enter for the full definition. (Opus, GPT-5.5, GPT-5.3 Codex) in Plan modeA mode that makes no edits: it researches the codebase and produces an editable plan you review before any code changes. Press Enter for the full definition. to generate and validate a plan, then hand the to-dos to a cheaper, faster coding model (ComposerCursor's own fast coding model, tuned for the editor and priced well below frontier models; the recommended day-to-day model for executing a plan. Press Enter for the full definition.) to execute.
Letting the frontier reasoning model write all the code burns the token budget fast - you don’t need it once the plan exists. Planning is itself token-efficient because the frontier model only reasons, it doesn’t output the whole implementation.
Cursor isn’t a desktop app piping your prompt straight to the provider. A proprietary cloud harness does optimization, context enrichment, caching and compaction, tuned per frontier model.
Via dynamic context discoveryThe agent pulling only the relevant parts of files, tools and MCP servers into context as needed, instead of loading everything up front. Press Enter for the full definition. plus caching, the harness reduces agent tokens by roughly 47% - even on other vendors’ models - so the same model can perform better here than in a thinner harness. When you cost a workflow, that reduction is already baked in.
The quality / latency / cost trianglepick two, on purpose
When a system-design question asks you to make a workflow “fast and cheap,” name the triangle and pick two on purpose for that specific workflow.
Then defend the third: “for an overnight migration I trade latency for quality and cost, batched on a large model, because nobody is watching the clock at 3am.”