AI Code Review: Why Reading the Diff Isn't Enough

AI code review is not the same discipline as reading a colleague's pull request. A junior engineer's bad code looks tentative and asks for help. AI-generated code looks finished whether it works or not, so the review has to shift from reading for style to verifying for truth: run it, test it, and put a second, fresh set of eyes on the diff before it ships.

I have watched a diff sail through review because it read cleanly: well-named variables, sensible structure, a docstring that explained its own intent. It failed on the first input nobody had typed into the demo. The code was not sloppy. It was wrong, and it looked exactly like code that was right. That gap is what this piece is about.

Key takeaways

If you read nothing else, read these.

• AI code review is a verification discipline, not a reading discipline. Run the code, test it, and check the result against a real signal before you trust it.
• Confident code is not correct code. AI-generated code fails the way any code fails, except it never looks unsure while doing it.
• Roughly 40% of code samples generated by GitHub Copilot in one security study contained exploitable vulnerabilities, across 1,689 programs and 89 scenarios tied to known weakness classes.
• The session that wrote the code is a poor judge of it. Review in a fresh context: a different person, or a second AI session that never saw the reasoning behind the diff.
• Automated AI review tools are strong on style and diff noise, weak on business logic and security reasoning. Treat them as a first pass, not the review.

Why AI code review is not the same as reviewing a colleague's PR

A human teammate's rough pull request usually signals where to look. A half-finished function, a comment like "not sure this handles nulls," a variable named tmpFixLater. An AI coding agent does not hedge that way. It produces a complete, confidently formatted diff whether the logic underneath is sound or not.

That confidence is the actual failure mode, not a side effect of it. A reviewer trained to scan for tone, formatting, and obvious gaps will pass code that looks done, because looking done is the only signal a traditional read gives you. AI-generated code is built to look done. It is not automatically built to be correct, and nothing in how it reads tells you which one you are looking at.

This is also why AI-Native review differs from AI-assisted review. When you are still driving and the model suggests a line, you catch mistakes as you type them. When the model plans, writes, and runs the whole change, you only see the finished diff, and every mistake it made along the way is already hidden inside a result that compiles.

The failure modes to check for first

Four patterns account for most of what slips through a casual AI code review. Check for these before anything else:

• Hallucinated APIs and packages. The agent calls a method or imports a library that does not exist, or exists under a different name, and the code looks idiomatic enough that nobody questions it until the build fails or a lookalike malicious package fills the gap.
• Silently dropped edge cases. Empty inputs, null values, and the unhappy path get handled everywhere the agent thought to look and nowhere it did not, with no comment marking the gap.
• Insecure defaults. Permissive CORS, disabled certificate checks, secrets read straight into a log line: the kind of shortcut that makes a demo work and a production system leak.
• Scope creep beyond what was asked. The agent "helpfully" refactors a neighboring function, changes a return type, or touches a file nobody asked it to open.

The security failure mode is not theoretical. Researchers tested GitHub Copilot against 89 scenarios tied to MITRE's top security weakness classes, producing 1,689 completed programs. Roughly 40% of them were vulnerable (Pearce et al., IEEE S&P 2022). That is not a rare edge case. That is close to half.

If you want a delivery team that treats this review discipline as the default rather than something bolted on after an incident, that is the model ViitorCloud's product engineering team builds around: verification wired into the workflow from the first commit.

Give the review something to verify, not just read

A read-through catches typos and obviously missing logic. It does not catch a function that returns 200 on a request it should have rejected. The fix is to stop asking whether the code looks right and start asking what it does when you run it.

Anthropic's own guidance on working with coding agents makes the same point from the other direction: give the agent, and the review, a check that produces a pass or fail signal, not an impression. "Claude stops when the work looks done. Without a check it can run, looking done is the only signal available" (Claude Code best practices). The same sentence describes a human reviewer skimming a diff.

Tests, a build that passes clean, a screenshot compared against the design, a log from a real run against staging data: each of these is a pass or fail signal a read-through cannot fake. If you cannot verify a change, the honest move is not to ship it on trust.

Review the diff in a fresh context, separate from the session that wrote it

The session, human or model, that just produced a piece of code is a poor judge of it. It is invested in the approach it already chose and reads its own reasoning back to itself as confirmation, not as a question.

Anthropic's documented pattern for Claude Code runs implementation and review as two separate sessions specifically to break that bias: one session writes, a second session reviews the diff having seen only the code and the criteria, not the reasoning that produced it. "A fresh context improves code review since Claude won't be biased toward code it just wrote" (Claude Code best practices).

I run this pattern with a second Claude Code session more often than I read a diff cold myself now. The second session has caught scope creep the first one talked itself into, changes to a file nobody asked it to touch, more than once. The catch is not that the second model is smarter. It is that the second model was never talked into agreeing with itself.

What AI code review tools catch, and what they miss

A layer of tools now sits between the agent and the merge button: GitHub Copilot's code review feature, CodeRabbit, Qodo, Greptile, and Claude Code's own /code-review skill, which runs a fresh subagent against the current diff and returns findings to the session that wrote it.

These tools are reliably good at the same things a strict linter is good at: style drift, obvious diff noise, patterns that do not match the rest of the codebase, missing error handling on paths the tool has seen many times before. That is real value. It is also a narrower job than "review this code."

None of them reliably replace judgment on business logic or security reasoning specific to your system. A tool can flag that a function looks unusually complex. It cannot tell you the discount calculation is wrong for your particular pricing model, because that requires knowing the pricing model, not the code.

Use them as a fast first pass that clears the mechanical noise, so the time a person spends reviewing goes to the part of the diff where a mistake costs something.

A checklist: what to read line by line, and what to test

Not every line deserves the same scrutiny. Spend human attention where a mistake is expensive, and let automated checks cover the rest.

• Read line by line: authentication and authorization logic, anything that touches money or billing, data handling and deletion, external API calls with side effects, and any change to a security boundary.
• Verify by running it: the happy path, the documented edge cases, and at least one input nobody specified, run against real or realistic data, not just the fixtures the agent generated for itself.
• Cover with automated checks: formatting, import hygiene, dependency versions, and regressions in code the existing tests already cover well.

If you only have time for one of the three, do the second. A test the agent's own fixtures pass proves the agent agrees with itself. A test built from an input you thought of independently proves something closer to the truth.

The AI code review trade-off nobody advertises

The entire pitch for AI-assisted coding is speed. Real review of AI-generated code, running it, testing it, getting a second opinion, is not fast, and pretending otherwise is how the failure rate above reaches production.

A 2025 randomized controlled trial from METR found experienced open-source developers were about 19% slower on real repository issues when using AI tools, despite believing the tools had sped them up by roughly 20% (METR, 2025). Sixteen developers, 246 real issues, a gap between felt speed and measured speed wide enough to change how you weigh your own team's self-reports.

Name the incentive honestly. If the whole case for AI coding is "ship faster," and real review measurably slows experienced developers down, the pressure runs toward skimming the diff and merging. That is exactly the condition under which a codebase with a roughly 40% vulnerability rate in generated samples reaches production unreviewed.

The revenue math only works if the review discipline survives contact with the backlog, not just the pilot. A team that ships fast and reviews badly is not fast. It is a team that has not been billed for its mistakes yet.

When to reject AI-generated code outright

Most of this piece is about fixing what an agent gets wrong. Some diffs are not worth fixing. Reject and regenerate, rather than patch, when any of these are true:

• You cannot explain what the code does. If neither you nor the agent can walk through the logic in plain language, you cannot evaluate it, and you should not merge it.
• It touches a security boundary you cannot verify. Authentication, authorization, and payment logic earn a hard no until someone can prove the change is correct, not just plausible.
• The fix count exceeds the rewrite count. Once you are patching the fourth issue in the same function, a fresh generation with a tighter spec is usually faster than continuing to patch.
• It passed by changing the test, not the code. An agent that "fixes" a failing test by loosening the assertion has not fixed anything. Reject on sight.

Frequently asked questions

How do I review code that AI wrote if I did not write it myself?

The same way you would review a stranger's pull request with no memory of the intent behind it: read it for what it claims to do, then verify that claim by running it. Do not assume the code is correct because the request or the description sounds reasonable. Assume nothing until a test, a build, or a real run confirms it.

Is AI-generated code less secure than code written by a human developer?

The best available data says it can be. One study found roughly 40% of GitHub Copilot's generated code samples contained exploitable vulnerabilities across 1,689 tested programs (Pearce et al., 2022). That does not mean every AI-generated line is unsafe. It means security review cannot be optional or skimmed.

Can an AI accurately review its own code, or does it need a second opinion?

It needs a second opinion. The session that wrote the code is biased toward the approach it already chose. A fresh session, given only the diff and the criteria, evaluates the result without that bias, which is why Anthropic's own Claude Code guidance runs implementation and review as two separate sessions.

What is the best AI code review tool right now: Copilot, CodeRabbit, Qodo, or Greptile?

None of them is a substitute for a person who understands your system, and all of them are useful for the same reason a linter is useful: they catch mechanical issues fast, before a human's attention is needed. Pick whichever integrates cleanly with your existing pull request workflow, then keep the line-by-line human read for the code sections in this piece's checklist.

Where this leaves you

AI code review is not a slower version of reading a pull request. It is a different discipline built around verification instead of impression: run it, test it, and let a second set of eyes, human or a fresh model session, look at the diff before you trust it. The tools help clear the noise. They do not replace the judgment.

I go deeper on what changes across the whole development lifecycle once the machine writes the implementation, not just the review step, in the AI-Native SDLC and my book The AI-Native SDLC.

If you are further along and working out how review holds up once an agent is pairing with you at pace, I cover that specific ground in AI pair programming.