Prefix caching: reuse the work you already did

Prefix caching: reuse the work you already didthe lever with Cursor's name on it

If two requests start with the same tokens, their KV cache for that shared span is identical. Prefix caching keeps those computed keys and values around and reuses them, so a request that shares a prefix skips prefilling it entirely. Prefill then starts at the first token that differs from the cached prefix and runs over everything after it.

Cursor's prompts are unusually repetitive, which is exactly the condition prefix caching loves. The same system prompt, the same tool definitions and often the same file or repo context get sent across many requests in a session. A high prefix-cache hit rate turns an expensive full prefill into a much cheaper cache read. On current provider pricing a cache read bills at a small fraction of the input rate, though the first write of a prefix can cost more than plain input.