How to Measure AI Coding Productivity and ROI

Track two pillars side by side: velocity and quality. A velocity number on its own rewards shipping faster even as the code degrades. Watched alone, quality has the opposite failure: it rewards caution even when nothing ships. This two-pillar model comes from a Cursor talk on developer productivity trends, which named time-to-marketHow fast a feature actually reaches customers; the outcome metric the velocity and quality pillars ultimately serve. Press Enter for the full definition. as the outcome both pillars serve.

Pillar one: velocity

The talk split velocity into three layers, which trade ease of measurement against closeness to the outcome.

The rows are ordered by what each one costs to measure, not by how much each one matters. PR velocityHow quickly pull requests are merged; the easiest delivery metric to measure, though it sits furthest from the customer outcome. Press Enter for the full definition. costs nothing to collect because your git host already counts it, so it is the cheapest place to start. Feature-completion velocity needs a shared definition of a completed feature, and someone has to write that definition down.

Start with the top row, and write down when you plan to move past it. If you open with feature completion because it is the better metric, you can spend a quarter defining terms and reach the review with nothing plotted. If you open with PR counts and never name a successor, a year later people are still defending PR counts.

Pillar two: quality

Quality has an external face and an internal one. Watch both, because more code from an agent can trade one for the other without anyone noticing.

The two faces change at different speeds. Customer-facing defects show up release by release, while coverage and maintainability shift over quarters. Judge both on the same short cadence and the slow internal metrics can make a good quarter look like a regression.

A small team should measure feature completion and customer-facing defects, and skip the layers in between. A large organization needs the middle layers too, because it has to compare teams that don't share definitions.

Story points travel worst of the three. They mean something only when the same people estimate the same way over time, which a five-person team with no estimation history can't offer. Large organizations have the opposite problem: every board has points, but each squad's points mean something different. Average twelve teams' story points and you have averaged twelve different units.

The bigger difference, in our view, is who reads the number. On a small team, the person measuring is also the person shipping, so a rough count checked monthly is enough to act on. Once the number goes to someone outside the code, people start comparing it, and comparison forces shared definitions.

Budget time for that argument before you promise the number to anyone.

Both are easy to count, and both mislead. The Cursor talk made the point directly: more AI-generated code doesn't necessarily mean better code or faster delivery to customers. It does mean more code to review and maintain.

"Active users" hides depth. The count treats a weekly visitor to the agent the same as someone whose whole workflow now runs through it. A login count measures reach, which is a smaller thing than adoption and easy to mistake for it.

If you report seats, count the people who used the agent for work they used to do by hand.

The failure is mundane. The number already exists, so it goes on the slide, and by the second quarter the slide is what people are managed against. Once a team is judged on how much code the agent wrote, the cheapest way to look better is to let the agent write more, and the review queue absorbs the difference.

Code half-life measures how long a piece of code lasts before someone has to change it. The Cursor talk offered it as a maintainability signal: code that stays stable for weeks or months is healthier than code that keeps churning.

Code rewritten within days of landing suggests the first version missed what was needed, or that the design couldn't carry the load. Treat that as something to investigate, whether a person or an agent wrote the code.

Check what kind of work a short half-life covers before you call it a quality problem. A surface nobody has shipped yet churns because its design is still being found, and config files churn by nature. Hold the task type steady, watch the same kind of change over two or three months, and the number starts to mean something.

The request itself is a bigger confound. A short half-life on agent-written code reads like a verdict on the agent, but it may be a verdict on the ticket. If vague tickets are the ones that go to agents, part of what you measure is which work was specified well, and the fix then belongs upstream, in how the task is written.

Cursor records much of this itself, so you don't have to infer everything from git. Your plan decides how much you see: the analytics dashboard is on Team and Enterprise plans, and the other tools here are Enterprise only.

Attribution is what makes Cursor BlameAn augmented git blame that records line-level human and agent co-authorship, so you can trace which code was written by AI versus a person. Press Enter for the full definition. useful for measurement. Once you know which lines an agent wrote, you can ask whether agent-written code carries more defects or churns faster than code people wrote. Expect the answer to vary by team. In our view it depends more on the task mix and on how well the work was specified than on who wrote the line.

On Enterprise, connect the Analytics API early, mostly for credibility. A number that lives only in the vendor's dashboard gets read as the vendor's number. Pull it into the same reporting stack as your delivery metrics and that objection goes away. The API's own page and Cursor's API overview list it for Enterprise teams, although Cursor's dashboard docs say every API except AI Code Tracking also works on Team plans.

Plan around what the audit log doesn't record. It covers administrative and security events and holds no prompts, agent responses or generated code, so Cursor's docs point to hooks for logging that activity. There is more on that record on how to measure AI developer productivity.

Cursor's November 2025 write-up describes a study by Suproteem Sarkar of the University of Chicago on its agent's early effects across tens of thousands of Cursor users. The study compared an "eligible" group of organizations, already on Cursor before its agent shipped, with a "baseline" group not using Cursor. Once Agent was made Cursor's default mode, the eligible group's rate of merged PRs rose 39% relative to the baseline group's time trend.

The organization count needs care. Cursor's write-up never gives one: its only "1,000" is a sample of users whose opening requests the study sorted by type. The 1,000-organization figure comes from LeadDev's coverage, which also says the 39% figure is based on 24 organizations that used Cursor, compared with eight similar companies that didn't.

What the study measured was a change of default. It says nothing about any one developer getting better, and it does not survive being restated as "39% faster", because output and elapsed time are different quantities.

Cite it as a case for making the agent the default, and stop there.

For measurement, the acceptance finding matters more. Cursor's write-up lists three theories for it and picks none: better use of rules and context, more confidence in judging agent output, or better-scoped tasks. The study also reports that experienced developers plan before coding more often. A Cursor talk offered planning as one possible explanation, but the write-up's three theories don't include it. If the link holds, the lever sits upstream of the model, in plan mode and in how the request is written.

Time-to-market acceleration: whether features reach customers faster than they did before. If they do and defects aren't climbing, the tooling is likely paying for itself, though other changes in the same period can move it too. If you can watch only one number, watch this one.

The Cursor talk warned that everything else can become noise. PR counts and acceptance rates are useful inputs, but they serve the goal of shipping value sooner. The failure to watch for is a team that improves them while the roadmap stalls.

When time-to-marketHow fast a feature actually reaches customers; the outcome metric the velocity and quality pillars ultimately serve. Press Enter for the full definition. doesn't move, find where a change waits before you blame the tooling: the review queue, or a release train that runs weekly whatever you do. Faster generation can't shorten a queue. A change that waits a week for a release slot loses whatever time the agent saved, and none of the numbers above will show it.

Cycle time looks like a substitute, and it nearly works. First commit to merge is cheap to pull, but it also moves with review load and scope changes, so much of what it registers has nothing to do with an agent. Keep it for when time-to-marketHow fast a feature actually reaches customers; the outcome metric the velocity and quality pillars ultimately serve. Press Enter for the full definition. surprises you and you need somewhere to start looking.