Benchmark Report: AI Coding Tools for TypeScript Teams

Decide what a fair test looks like before you run the benchmark. If you decide afterwards, the numbers will favor the tool you already prefer.

Fix the task mix before you see any results. If you choose which tasks count after the test has run, you will pick the ones that agree with what you already thought, without noticing. Without a written plan, a benchmark is an anecdote with a chart attached. Write down the stack, the task types and the sample size before the first run.

Every benchmark has limits. Naming them is what makes the rest of the report credible, so state plainly what the result does not cover.

• Sample size, repo type and task mix.
• Models, tool versions and seat cost used.
• Review time added by AI-generated changes.
• Where the result should not be generalized.

Put each limit next to the result it qualifies, not in a footnote. A result shown without its sample size invites readers to apply it to their own repo.

Cursor ships often, so advice about it can go out of date. The table summarizes Cursor's recent releases, so you can check this page against the current product. For each release's date and sources, see What's new in Cursor 2026.

Because they fail at different scopes. A test tells you one package still behaves the way you asked. The typecheck tells you the rest of the repo still compiles against the change. Roll them into one pass or fail, and you can't tell which of the two a tool cleared.

Take a patch that lands inside one workspace package. Its own tests go green, but a dependent package no longer builds, because an exported type quietly narrowed. If the harness runs only that package's test command, a tool that broke other packages scores as well as one that didn't. That is the opposite of what you wanted to measure. So run the repo-wide typecheck as its own gate, at the cost of one more command, and record both results for every task.

This one choice can make two published TypeScript comparisons disagree. Two teams can run near-identical prompts against near-identical repos and name different winners, because one checked the whole project and the other checked a folder. If neither report says which, the two results can't be reconciled.

Use the typecheck command your CI runs. If CI builds with project references rather than one flat pass, a hand-rolled command measures a configuration nobody ships. It can also pass code that CI would reject, so every score comes out higher by an amount you can't see.

Run both checks on every task, even when the first one fails. A patch that breaks compilation can also crash the test runner, and the crash looks like a test failure. With both results recorded, you can see the type error behind it instead of filing it as a behavior bug.

Four types, run in equal numbers. The method above says to fix the task mix before you see any results. This is that mix for a TypeScript repo, with the reason each row is there.

Keep the fourth row, even though it is the hardest to set up. A version bump produces many errors with one underlying cause, which is the kind of work agents are supposed to handle well, and small, self-contained tasks never test it. Score the second row against the code before the patch. A new test that passes on the old code proves nothing about the change, so run it once without the patch and confirm it fails.

Keep the repo and the prompts identical across tools, down to the wording.

If one tool needed a rewritten prompt to get enough context, that is a finding about the tool. Log it as a finding instead of fixing the prompt in place.

We would leave greenfield tasks out, though the choice is arguable. You are buying help inside code that already exists. A from-scratch file has no local patterns to break, so every tool looks competent on it.

Read each diff for escape hatches. A type error can be fixed or muted, and both turn the typecheck green. Look for an any, a double assertion through unknown, a return type widened until the mismatch disappears, and a @ts-expect-error parked on the line that used to fail. Each one passes the harness and leaves a regression in the repo.

So an automated score can't carry this report alone. The cheap check is a grep over each diff for those four patterns, plus one reviewer note per task. The expensive check is a second reviewer who never learns which tool produced which patch. If you can afford it, blind the patches, because knowing the vendor changes how generously people read a diff, in either direction.

Pin the checks too. A tool can edit the test file as well as the source, so a green result proves little if the tool could have changed the check. Write the test and the typecheck command before the run, keep both outside the tool's writable scope, and score against the pinned copy.

Pinning also settles an argument you would otherwise have at the end. When one tool's patch fails a pinned test, somebody will suggest the test was wrong. Sometimes it is, so decide that question before you know whose patch failed.

Less than the method implies. With five engineers you won't reach a sample size that supports a percentage, so don't publish one. Run two tools over the same four tasks, read all eight diffs yourself, and write down which ones you would have merged.

For a small team, that is a better use of a week than a wider matrix. The decision is small and reversible: you are picking a default for a team you can talk to in one room, and you can change it in a month.

A large org has the opposite problem. Sample size stops being the constraint, and representativeness takes its place. Run the benchmark in the pilot team's tidy service repo, and you learn nothing about the monorepo where most of your review time goes. The TypeScript workflow guide covers the package-boundary habits that make that repo the hard case.

So name the repo the numbers came from, next to the results. A reader can only apply your result to repos like the one you tested, and without the name, each reader will assume it was their kind of repo.

Add reviewers with care when you scale up. Ten tasks per tool read by one reviewer can tell you more than a hundred tasks scored by a rota. Each extra scorer brings a slightly different standard, and more runs don't remove that variance.