How to Build a Pull Request Review Agent

Building pull request review agent starts with a clear trigger and ends with a handoff a person can check. The steps below define that path, and the code shape under them shows how one run holds together.

1. 1Define the trigger: issue, PR, failed CI job, docs change or support ticket.
2. 2Limit tool access to the files and systems the agent needs.
3. 3Add a plan step before writes.
4. 4Run checks and produce a short reviewer handoff.
5. 5Log the prompt, changed files, commands and result.

type AgentJob = {
  task: string;
  context: string[];
  allowedTools: string[];
  checks: string[];
  handoff: "diff" | "comment" | "pull_request";
};

Start with allowedTools. I'd say it's the field people end up regretting, though I'll admit that's a guess from a small sample. The pull is to grant broad access while you're building, because narrowing it is fiddly and everything works either way when you're sitting there watching. An agent running unattended with repo-wide write and a shell is a genuinely different risk than one that can edit three paths and run a single command, and you find out which one you built on the run where something goes wrong.

handoff is what decides whether anyone trusts the thing. Returning a diff keeps a person in front of it before anything lands, which is the right default while you're still learning how the agent behaves on your codebase. Opening a pull request is that same gate with better ergonomics, once you've earned the confidence. A comment is the lightest option and suits agents that only diagnose. What none of them should do is merge — not until the commands in checks are ones you'd have run yourself anyway.

Most failures here are not exotic. They come from giving the agent too much room, or trusting a "done" that no test ever proved.

Scope bites earliest. Give an agent a vague task and it will go and find related work to do, which reads as initiative right up until you're staring at a diff across twenty files with no idea which changes were the point. A better model doesn't fix this. A narrower task does: name the files, name the behaviour that should change, say what's out of bounds.

Weak checks bite latest, and hardest. An agent reports success from its own reading of its own work, and it isn't lying when it does that — it just has nothing external telling it otherwise. Maybe the better way to put it is that if the only thing between the agent and your main branch is its own summary, you don't have a review step at all, you have a formality with a nice interface. Give it a command whose exit code you trust, and let the run fail when that command fails.

Build the part that is yours. Generic bug-finding on a diff is a shipped product and you are unlikely to beat it as a side project. What no vendor knows is your routing: which change needs which human, and what counts as risky in this repo.

The most useful version of this build is a classifier more than a reviewer. A PR-opened automation rates risk, low-risk changes move without extra approval, and the higher tiers get reviewers assigned from git blame so the change lands in front of the people who wrote the code being changed. That last step is the one a general-purpose reviewer cannot do for you, because it needs your repo history and your idea of who owns what.

The second reason to build one is delivery. Findings that live only in a comment thread go stale, so a tool or MCPModel Context Protocol. A standard that lets an AI agent pull in context from outside the repo, like Jira tickets or internal docs. Press Enter for the full definition. that files them into Slack or your issue tracker is a small piece of plumbing that decides whether anyone acts on them at all. Not glamorous. Probably the part of this build that pays back fastest.

A short comment pointing at the risk and at the proof, and by default nothing else. An agent that rewrites the PR hands you a second diff to review, which is a net loss dressed up as help.

Give yourself a way to tell whether it is working. Cursor's Security Agents track findings raised, findings fixed and a resolution rate, and they settle "fixed" by having a model read the incremental diff after the next push. Copy that shape even crudely: log the finding, look at the next push, count what got addressed. A review agent whose findings nobody resolves is a cost centre with good intentions.

The number I would watch first is the share of comments that produce a change, not the count of comments.

PR-opened is the default worth starting from, because it catches everything and does not depend on anyone remembering. A pre-push run is cheaper and skippable, which makes it a good habit and a bad guarantee.

Whichever you pick, filter it. Automation triggers take allow-list filters, so you can scope a run to specific people, channels, a keyword or a regex instead of firing on every draft PR in the repo.

There is a governance line here that matters more than the code does. An automation should never merge on its own; it produces a human-reviewed PR, and the person clicking merge is accountable for it. And the genuinely new exposure is not what the agent can do, it is who can set it off, so do not wire a code-touching automation to an audience you would not hand repo write access to.

And if you build this on the SDK rather than on Automations, local.autoReview routes local tool calls through a classifier instead of letting them bypass review. Treat it as a gate, not as evidence that a human pass was unnecessary.