How to Automate Pull Request Reviews

A pull request review is the check a code change goes through before it merges: a second reader works through the diff, looks for bugs, security problems, and broken contracts, and either approves the change or sends it back with specific comments. It is the last point where a mistake is cheap to fix, because after the merge the bug ships with everything built on top of it.

For a small team, review time is the scarcest resource in the pipeline. The people qualified to review are the same people writing the code, so reviews queue behind feature work, get rushed at the end of the day, or get skipped because the change looks simple. Every skipped review is a bet that nothing in the diff will bite later.

Why unreviewed pull requests quietly cost you

When nobody owns review, the process decays in a predictable order. First the depth goes: reviewers skim the diff, comment on naming, and approve without tracing what the change actually touches. Then the coverage goes: small PRs merge unread because "it's simple", which is exactly the reasoning that lets a nil dereference or a leaked behaviour change through. Finally the habit goes, and review becomes something the team does when there happens to be time.

The costs surface later and land somewhere else. A broken API contract shows up as a support ticket. A missed N+1 query shows up as a slow page under load. A string-interpolated SQL statement shows up as a security incident. None of these trace back to the ten minutes of review that would have caught them, so the process never gets the blame.

What the manual process looks like

Done properly, a pull request review is a ritual with five steps:

1. Read the diff and work out what the author intended and how far the change reaches.
2. Walk the correctness checklist: exceptions and nil dereferences, off-by-one errors, unhandled error returns, expensive work on the request path, behaviour changes that leak into other components, and anything that breaks an API or serialized format without a migration path.
3. Check the tests. Every behaviour change needs tests covering the edge and error cases, and they have to assert behaviour rather than implementation. A test that passes the instant it is written proves nothing.
4. Run a security pass: trace every input the change touches back to its source and ask whether an attacker controls it, rather than flagging on pattern-match alone.
5. Write it up as comments anchored to a file and line, phrase uncertainty as a question, and end with one verdict.

Each step rewards patience and consistency. Together they take real time per PR, which is why the checklist shrinks on busy days and disappears on the busiest ones.

What an agent can automate

Most of that ritual is systematic reading, which makes it a good fit for an agent running a fixed workflow:

• Read the diff in context. The agent reads the pull request diff, the changed files, and the surrounding code from your connected repository, then summarises the author's intent and the scope of the change before judging anything.
• Run the correctness and design pass. It works the full checklist: worker errors, performance problems like unbounded loops and N+1 queries, side effects that leak to other components, and backwards-compatibility breaks. It traces the callers of any contract that moved and confirms behaviour changes are covered by tests that assert behaviour, not implementation.
• Run a confidence-gated security pass. The agent traces attacker-controlled input to its sink before flagging anything, skips framework-mitigated and server-controlled patterns, and reports only high-confidence, exploitable findings. A clean pass is a real result, not a failure to find something.
• Self-check the findings. Before drafting, it drops low-confidence security flags, drops pure style nits, and confirms every surviving comment is anchored to a file and line and is actionable.
• Draft comments and a verdict. Each finding is classified Critical, Important, or Minor, states what is wrong and what to do instead, and the review ends with one verdict: Approve, Approve with changes, or Request changes.

Two things stay with people. The verdict is drafted, never posted, until you approve it. And risky territory is escalated rather than waved through: changes touching a database schema, an API contract, a new dependency, a performance-critical path, or security-sensitive code get flagged for a senior reviewer.

The guardrails that make it safe

An automated reviewer fails in two ways: it posts noise that trains the team to ignore it, or it posts confident nonsense that erodes trust in review itself. The guardrails target both.

Against noise, the workflow includes an explicit self-check step where the agent reviews its own findings and cuts everything low-confidence, framework-mitigated, or stylistic before a human ever sees the draft. The security pass treats a false positive as the cardinal sin and investigates before flagging.

Against unsupervised posting, the workflow ends at a human approval step. The agent never posts anything itself. The drafted comments and verdict wait in your inbox, you edit anything that reads wrong, and nothing reaches the pull request without your sign-off.

Set it up in Task Machine

The Pull request review playbook installs the method above as working records in your workspace: the Reviewer Agent, the Code review workflow with the approval step built in, the three skills carrying the review checklist, the security pass, and the feedback discipline, and a standing goal that keeps every PR covered. Setup takes a few minutes. You need a Task Machine workspace and permission to install playbooks (workspace owners have it). Repository access is not required up front; until you connect it, the agent reviews from the diff and files you attach to each run.

1. Find the playbook

Open Playbooks in your workspace and search for "pull request review", or browse to the Engineering category. The card lists what the playbook creates and the models its agent runs on.

2. Preview what it installs

Preview & install opens the full contents before anything is created: the Reviewer Agent, the Code review workflow, the code-review, security-review, and requesting-code-review skills, and the Every PR reviewed goal. The preview also names the one requirement, a connected repository, and notes that the agent works from attached diffs until access is connected.

3. Point the reviewer at your repository and standards

Start setup asks for the details that shape every review. Repository picks the repository the agent reviews. Review focus areas lists the areas you want weighted in every pass, such as accessibility or performance. Project standards or constraints describes the conventions every comment should respect, so the agent flags violations of your rules rather than generic style. Verification command names the check a change is expected to pass, so the agent can ask whether it was run.

4. Generate and review

Generate customized playbook bakes your answers into the agent instructions and the workflow prompts. The result comes back for review before anything is created. Read through the agent and workflow cards and confirm the standards and focus areas you wrote appear where the agent will act on them.

5. Install

Install customized playbook creates everything in one step and lists what landed in your workspace. One follow-up arrives in your inbox: Start Code review, which asks you to confirm the repository, the pull-request source, the review standards, the security pass, and the comment-approval step before the agent reviews its first diff. From then on, each pull request kicks off the workflow: the agent reads, reviews, and self-checks, and the drafted comments and verdict wait in your inbox for approval before anything is posted.

What good looks like

The bundle's own standard for a great review is the honest benchmark: a tight, line-anchored, severity-classified set of actionable comments with a clear verdict and no false-positive noise. Three signals tell you the process is working:

• Every PR carries an approved review before it merges. That is the standing goal the playbook installs, and the gap between PRs opened and PRs reviewed is the first number to watch.
• Comments survive contact with the author. If findings routinely get dismissed as noise, the self-check is not doing its job. Each comment should name a file, a line, what is wrong, and what to do instead.
• Escalations happen where they should. Schema changes, API-contract changes, new dependencies, performance-critical paths, and security-sensitive code should reach a senior reviewer flagged, not approved quietly.

Common questions

Will the agent post review comments on its own? No. The workflow ends at a human approval step. The drafted comments and verdict wait in your inbox, and nothing is posted to the pull request without your sign-off.

Does this work before the repository is connected? Yes. Until repository access is connected, the agent reviews from the diff and files you attach to each run. Connecting the repository lets it read the changed files and surrounding code itself.

Will an automated reviewer flood PRs with nitpicks? The method is built against that. The agent does not block on style, and a self-check step drops pure style nits and low-confidence flags before you see the draft. What survives is classified Critical, Important, or Minor, so the author knows what must be fixed and what is a note for later.

What happens with risky changes like a schema or API change? The agent stops and flags them. Changes touching a database schema, an API contract, a new dependency, a performance-critical path, or security-sensitive code are escalated for a senior reviewer instead of being waved through with a verdict.

Is a review with no security findings a bad review? No. The security pass reports only high-confidence findings where attacker-controlled input reaches a vulnerable pattern, because a false positive costs more trust than it saves. A clean pass after that investigation is a real result.