1 min lesson
Caching, batching, prompt reuse
Rebuild the parts of "Caching, batching, prompt reuse", then say why each one matters.
Step 1 of 3
Caching, batching, prompt reusecut cost and latency together
- Prompt / prefix caching
- Reuse the cached prefix of rules, system prompt and stable context so repeated runs only pay for the new tokens.
- Result caching
- Identical input + revision returns the prior result instead of re-calling the model.
- Batching
- Group independent calls, such as one per file in a refactor, to amortize overhead and raise throughput.
Learn more
Full explanation
Token budgeting
Token budgetingmeasure before you ship
You can't defend a cost you never measured. Instrument tokens per task during the build, then set a ceiling so a pathological input can't quietly 10x the bill in production.
Log input and output tokens per task and watch the p50 and p95, not just the average. The p50 is the typical task and the p95 is where the costliest 5% of tasks begin.
Set a per-task ceiling that aborts or downgrades the model before spend runs away, the same way you cap steps in the loop.
Learn more
Full explanation
Optimize for the right tokens, not the fewest
Optimize for the right tokens, not the fewesttrue efficiency = efficiency × quality
The target is the cost of a finished task. A token-cheap model that needs 100 re-prompts wastes more than it saves, and so does a model that one-shots a trivial feature while burning a billion tokens. Tokens map straight to dollars, since each model has distinct input, output and cached rates, so small inefficiencies compound at scale. Good hygiene produces cost efficiency and a better experience as a side effect. Spend the right tokens on the right step.
The flagship loop: use a frontier reasoning model in Plan modeA mode that makes no edits: it researches the codebase and produces an editable plan you review before any code changes. Press Enter for the full definition. to generate and validate a plan, then hand the to-dos to a cheaper, faster coding model such as Composer 2.5The current Composer release, better at long-running tasks and at judging when a job needs a light touch versus deep work. Press Enter for the full definition. to execute.
Letting the frontier reasoning model write all the code burns the token budget fast, and you don't need it once the plan exists. Planning is itself token-efficient because the frontier model reasons and leaves the whole implementation to the cheaper model.
Between your prompt and the provider, a proprietary cloud harness handles optimization, context enrichment, caching and compaction, tuned per frontier model.
On Cursor's own measure, dynamic context discoveryThe agent pulling only the relevant parts of files, tools and MCP servers into context as needed, instead of loading everything up front. Press Enter for the full definition. plus caching reduces agent tokens by roughly 47%, even on other vendors' models, so the same model can perform better here than in a thinner harness. Measure your own tokens per task; do not promise a customer the 47%.