How to Build a CI Failure Fix Agent

Building CI failure fix agent starts with a clear trigger and ends with a handoff a person can check. The steps below define that path, and the code shape under them shows how one run holds together.

1. 1Define the trigger: issue, PR, failed CI job, docs change or support ticket.
2. 2Limit tool access to the files and systems the agent needs.
3. 3Add a plan step before writes.
4. 4Run checks and produce a short reviewer handoff.
5. 5Log the prompt, changed files, commands and result.

type AgentJob = {
  task: string;
  context: string[];
  allowedTools: string[];
  checks: string[];
  handoff: "diff" | "comment" | "pull_request";
};

Start with allowedTools. I'd say it's the field people end up regretting, though I'll admit that's a guess from a small sample. The pull is to grant broad access while you're building, because narrowing it is fiddly and everything works either way when you're sitting there watching. An agent running unattended with repo-wide write and a shell is a genuinely different risk than one that can edit three paths and run a single command, and you find out which one you built on the run where something goes wrong.

handoff is what decides whether anyone trusts the thing. Returning a diff keeps a person in front of it before anything lands, which is the right default while you're still learning how the agent behaves on your codebase. Opening a pull request is that same gate with better ergonomics, once you've earned the confidence. A comment is the lightest option and suits agents that only diagnose. What none of them should do is merge — not until the commands in checks are ones you'd have run yourself anyway.

Most failures here are not exotic. They come from giving the agent too much room, or trusting a "done" that no test ever proved.

Scope bites earliest. Give an agent a vague task and it will go and find related work to do, which reads as initiative right up until you're staring at a diff across twenty files with no idea which changes were the point. A better model doesn't fix this. A narrower task does: name the files, name the behaviour that should change, say what's out of bounds.

Weak checks bite latest, and hardest. An agent reports success from its own reading of its own work, and it isn't lying when it does that — it just has nothing external telling it otherwise. Maybe the better way to put it is that if the only thing between the agent and your main branch is its own summary, you don't have a review step at all, you have a formality with a nice interface. Give it a command whose exit code you trust, and let the run fail when that command fails.

Partly, and it is worth checking before you write a line of this. Cursor's cloud agents automatically try to fix CI failures in pull requests they opened themselves. That route supports GitHub Actions only, and the automatic fixing is available on Teams today, with support for non-Teams accounts described as coming soon.

The dividing line is whose commit broke the check. Agents do not auto-fix CI failures on human commits, so the failures a platform team tends to care about, the red checks on branches people pushed by hand, sit outside the built-in path entirely. That is the space a build of your own is actually for.

You can also just ask. Tagging an agent in a comment with @cursor please fix the CI failures works on the pull request in front of you, and @cursor autofix off switches the automatic behaviour off for that one PR.

So the order I would put this in: a week on the built-in path against real red pull requests, then the build. Reading a handful of the agent's attempts tells you which of your failures are tractable at all, and that is the one thing designing the loop first cannot tell you. Skip it and the usual result is a well-built repair agent aimed at a class of failure that needed a person anyway.

Give it a ceiling and a list of conditions where it declines to act at all. Cursor's own auto-fix stops after 10 CI-failure follow-ups on a pull request, and it skips the attempt when the same check is already failing on the PR's base commit, when a human pushed a new commit, or when someone sent the agent a follow-up message.

The base-commit condition is the one worth copying first. Actually, that is too soft: I would not ship the loop without it. A check that was red before your branch existed is not your branch's problem, and an agent that cannot tell the difference will keep editing until the check turns green. The patch you get in that state is worse than no patch, because it changes code that was never broken and buries the real fix somewhere inside the diff.

Ten is not a magic number. Any small ceiling works, so long as the run ends by saying it gave up rather than by going quiet.

Now, the human-commit rule guards against a quieter case. Two authors on one branch, one of them a machine. If someone is halfway through their own fix, an agent patching underneath them produces a conflict neither of them wrote. Which is a small cost the first time, and the reason people switch the whole thing off the second time.

The ones where the log names the problem and any engineer on the team would fix it the same way. A type error, a lint rule, a snapshot that needs regenerating, an import still pointing at a file that moved.

Flaky tests are the case to keep out, and the reason is a bit uncomfortable. Ask an agent to make a failing test pass and it will find a way to make it pass. On a real bug that is exactly what you wanted. On a race condition it means a retry wrapper, or a quietly widened assertion, and you have automated the removal of a signal you were paying to have.

Cursor's marketplace lists a template for triaging failed GitHub Actions workflows and routing the result into a reviewable follow-up. Worth reading before you write your own, if only to see where somebody else drew the line between investigating and changing things.

For the loop itself, agent -p "..." is the headless form: a prompt in, a result out, no interactive UI, which is the shape that fits a CI job where nobody is at a keyboard to approve a step. Give it an explicit allow or deny policy for shell commands rather than the permissions you happen to use interactively.

And start it as a diagnosis-only agent. Comment on the failure, name the suspected cause, propose no patch. A month of reading those comments is the cheapest way to learn whether the patches would have been any good.