How to Measure AI Developer Productivity

A benchmark is only as honest as its method. Decide what a fair test looks like before you run it, or the numbers will just flatter the tool you already prefer.

Decide the task mix before you see any results. This sounds like pedantry and isn't: choose which tasks count after the test has run and you'll pick the ones that agree with what you already thought, and you'll do it without ever noticing you did. Writing down the stack, the task types and the sample size in advance is most of what separates a benchmark from an anecdote with a chart attached.

Every benchmark has limits, and naming them is what makes the rest trustworthy. Be explicit about what the result does not cover.

• Sample size, repo type and task mix.
• Models, tool versions and seat cost used.
• Review time added by AI-generated changes.
• Where the result should not be generalized.

Cursor ships often, so here is the current state of the surfaces this page touches. Each row links to the source where you can confirm the detail.

Two analytics signals, read together, tell you who is getting value and where adoption is stalling: request volume and acceptance rate of committed code. Plot them as a quadrant.

The ROI ladder

Proving value is a phased climb. Don't jump to dollars on day one.

Don't reuse the engineering yardstick. Lines of code and PR count measure code authoring; they undercount roles whose value is reach and depth rather than volume. For those roles, measure how much more of the work now gets the full treatment.

The honest framing he gives: "I've never met a security team where every single PR at the company is reviewed in security depth." The metric that matters is the share of changes that now get that depth, not the wall-clock of any one review.

More than most teams realise, and one thing you cannot. Cursor's team dashboard reports AI share of committed code, agent edits, Tab completions and active users across Agent, BugbotCursor's automated PR reviewer that posts inline findings and can push fix commits from isolated VMs. Press Enter for the full definition., cloud and the CLI. Cursor BlameAn augmented git blame that records line-level human and agent co-authorship, so you can trace which code was written by AI versus a person. Press Enter for the full definition. extends git blame with line-level human and agent co-authorship, exposed as an AI-code-tracking API.

Conversation InsightsA Cursor analytics view that passively categorises what agents are doing (new features, bug fixes, refactors) so leaders can see where engineering time goes. Press Enter for the full definition. gets overlooked in that list. It categorises work as new feature, bug fix or refactor with nobody labelling anything, and it can flag under-specified agent turns that would have gone better through Plan modeA mode that makes no edits: it researches the codebase and produces an editable plan you review before any code changes. Press Enter for the full definition. first. A read-only Analytics API pulls all of this into your own reporting stack, so none of it has to arrive as a screenshot.

What none of that gives you is the prompt. Audit logs cover administrative actions, not agent responses or generated code, so "what did we ask for and what came back" is the part you have to instrument yourself. The documented route is hooks: a prompts-submitted hook fires when a prompt is sent, a code-generated hook when code is written, both posting to an endpoint you run, with Cursor's own advice being to log metadata rather than full content.

Worth deciding early whether you want that record at all. It answers real questions about where people are struggling, and it is also a log of how your colleagues think, so the metadata-only version is the one that tends to survive being explained to the team it describes.

Your own trend, on a handful of task types you ship repeatedly. Not a public benchmark, and not the published research, although the research is worth knowing about: a University of Chicago study of 1,000 organisations adopting Cursor found a 39% increase in org-level output, with teams merging 39% more pull requests after Agent became the default mode.

You are not going to reproduce that internally. One team, one quarter, no control group, and every other variable in the company moving at the same time. Measuring four repeated task types before and after is a much weaker claim than a study of a thousand organisations, and it is the one you can actually defend in a room.

There is one finding in that study I would carry straight into your method, though. More-experienced developers were more likely to accept agent-generated code, plausibly because they plan more before they start. So an acceptance-rate comparison across a mixed-seniority team is measuring seniority as much as tooling, and a league table built on it reads a seniority distribution as a skill gap.

Read it per cohort, or read each person against their own last quarter. Ranking people against each other on that number is the version that comes back to bite you.

AI share of committed code gets promoted to the target, mostly because it is the easiest number to screenshot. An agent can author a large fraction of a codebase that never made a customer's life better, so a rising share tells you who authored the lines and very little else.

It also climbs when nothing ships.

Put something beside it that is hard to game and costs nothing to collect: how long code survives before it has to change. Code half-life, if you want a name for it. Code rewritten within days suggests the first pass missed the problem, whoever or whatever wrote it.

No single number belongs in a leadership deck without a counterweight next to it, and the counterweight has to be one you would find awkward to present. If the pair moves in opposite directions two quarters running, I think what you have been measuring is the tool's output and not the team's.