How to Use a Failing Test as an AI Coding Spec

The working pattern for failing test as an AI coding spec is small enough to review and specific enough to repeat. Give the agent a named task and only the context it needs, then tie the result to a check you can rerun. The table below breaks that into moves.

Each of those moves fails in its own particular way, and the order they come in is doing more work than it looks like it is. Skip the boundary and you get a diff nobody wants to read, because the agent has quietly rewritten files you never meant to open. Thin context fails more quietly: the code compiles, the tests pass, and it ignores every pattern the rest of the repo follows. Then there's the check, which to me is the real one, because it's the whole difference between a result you verified and a result you're taking someone's word for.

The step people skip is the plan. I think it's because it feels like overhead, thirty seconds of nothing visibly happening while you're trying to get work done. Actually, that's not quite the reason. It's that the cost of skipping it lands much later, so it never registers as the mistake it was. If the plan names files you didn't expect, you've learned something for free. If it names the right ones, you've got a reference to check the diff against when it arrives. And once there are four hundred lines on the screen, changing the approach means throwing that work away, which nobody is good at.

Yes. The frame below is a starting point, not a script. Fill in your own files, constraints and done state, and keep the plan step so the agent commits to an approach before it writes code.

Task: [one outcome]
Context: [files, errors, docs and examples]
Boundary: [what not to touch]
Done when: [test, typecheck, screenshot or review proof]

Before editing, write a short plan with files, risk and checks.

The Boundary line is the one people leave out, and probably the one doing the most work. Without it every file the agent can reach is fair game, so you end up reviewing incidental edits to config and imports and some helper you'd forgotten existed, all mixed in with the change you actually wanted. Naming what not to touch takes a few seconds. Reverting it afterwards does not.

"Done when" has to name something you can run. A test name, a typecheck, a command whose output you can read, a screenshot of one specific state. "Done when the bug is fixed" doesn't count, and I'd say that's the single most common version of this mistake, because it quietly hands the judgment back to the agent, and the agent is going to tell you it's finished either way.

Running failing test as an AI coding spec as a team comes down to one habit: leave a trail the next reviewer can follow. The steps below keep the prompt and its proof attached to the change, so nobody has to reverse-engineer what the agent did.

1. 1Pick one real backlog item with a clear owner and expected result.
2. 2Add only the context the agent needs: files, failing output, constraints and done state.
3. 3Ask for a plan before code when the task touches more than one file.
4. 4Run checks that match the risk: unit test, typecheck, visual pass or review checklist.
5. 5Capture the prompt, diff, result and reviewer note so the workflow can be repeated.

Keep whatever lets you rerun the work or hand it to someone else. A finished task is the merged code plus the short trail that explains how it got there.

• The prompt or plan that shaped the work.
• The files changed and the reason each file changed.
• The command, screenshot or review note that proved the result.
• The rule, checklist or template you would reuse next time.

Plan modeA mode that makes no edits: it researches the codebase and produces an editable plan you review before any code changes. Press Enter for the full definition. does search-and-retrieval across the repo, then writes a large Markdown plan you and the agent refine before any code. It tolerates a vague, ill-formed prompt and uses the agent to check your assumptions and build something more robust. Treat that plan as a Markdown rough-draft - a spec intermediary between intent and implementation.

For test-driven work in the age of agents, run it as two plans. First, plan and write a skeleton of tests that pins the functional requirements. Then write a second plan to implement the logic that turns those tests green. Agents do their best work when requirements are specified up front, so a test skeleton makes an ideal guiding light.

Because it has already shown that it can fail. The old discipline is to never trust an automated test you did not see fail, and a red test clears that bar before you have written a line of the fix. A written spec cannot clear it at all. A green test nobody has seen go red might be asserting nothing, and you would have no way to tell.

What a failing test is not, though, is complete. It is a spec with exactly one example in it, and a patch that satisfies one example is not necessarily the behaviour you wanted. That is fine when you know the assertion is representative. It stops being fine when the test covers a happy path and the bug lives two branches over.

Put the test file in the boundary, not in the scope. The check and the thing being changed cannot be the same file, or "done when the test is green" stops carrying any information at all. Say which file the fix belongs in, and say the test is not to be touched.

Sometimes the test is the thing that is wrong. That is a real task and a separate one, and probably the more common of the two on a suite that has drifted. Decide which job you are doing before you prompt, because a run that changed both the assertion and the implementation is a run you cannot draw a conclusion from.

Read the changed-files list before you read the result. If the test moved, the green means nothing.

One. With two failures in play you cannot attribute the patch to either of them, and the agent will chase both at once, which is how a small fix arrives spread across several files.

Give the loop its own session as well. The guidance out of Cursor's QA workshop is to keep the context window as empty as you can for test work and to open a fresh agent that only runs and writes tests, because in a long conversation the rule context can get overwritten as the window fills and the agent quietly stops obeying it. Compact at roughly 50 to 60 percent of the window rather than waiting for the ceiling.

Which is easy to miss, because a rerun loop runs long without ever feeling long.