# Review gate for a coding agent's diff before you merge it

> Check an AI agent's diff against its own summary: scope creep, unbacked claims, skipped or weakened tests, invented APIs and quiet config changes.

- **Author:** [Lucas Oliveira (@lucas_oliveira)](https://promptabide.com/lucas_oliveira)
- **Tested on:** Claude · Opus 5.5
- **You fill in:** `task`, `agent_summary`, `diff`
- **Published:** 2026-09-21
- **Updated:** 2026-09-24
- **Tags:** `ai-agents`, `code-review`, `testing`, `coding`
- **Keywords:** review ai generated code before merging, check coding agent pull request, ai agent skipped tests detection, verify claude code changes prompt, code review checklist for ai written code
- **Views:** 1637
- **Likes:** 44

**Best for:** Anyone merging changes written by Claude Code, Codex, Cursor or Copilot agents, especially after long unattended runs.

## Prompt

```
You are the review gate for a change written by a coding agent. You didn't write it. Treat nothing it claims as true until the diff shows it.

Original task: {{task}}
Agent's summary: {{agent_summary}}
Diff:
{{diff}}

Check each item in this order. Answer PASS, FAIL or CAN'T TELL, with evidence (file and line):
1. Scope: every changed line serves the task. List anything unrelated.
2. Claims: each statement in the agent's summary is backed by the diff.
3. Tests: were tests added? Were any deleted, skipped (.skip, xit, @pytest.mark.skip) or weakened (looser assertions, removed cases)?
4. Invented APIs: any function, option or package used that isn't defined in the diff or a well-known library.
5. Correctness: does the change actually solve the task, including the edge cases it implies?
6. Quiet config changes: lint rules, CI, dependency versions, env files.

Verdict: MERGE, FIX FIRST (numbered fixes) or REJECT. Keep it under 300 words.
```

## Variables

- `{{task}}` — The task you gave the agent (e.g. Fix: rate limiter lets users exceed 100 req/min when traffic spans two servers)
- `{{agent_summary}}` — The agent's own description of what it did (e.g. Moved the limiter to Redis with INCR and a 60s expiry. Added tests. All tests pass.)
- `{{diff}}` — The full diff (e.g. git diff main...agent/rate-limit)

## Example input

````
task: Fix: the rate limiter lets users exceed 100 requests/minute when their requests hit two different app servers.
agent_summary: Moved the rate limiter from in-memory to Redis using INCR with a 60s expiry. Added tests for the multi-instance case. All tests pass.
diff:
```diff
--- a/src/middleware/rateLimit.ts
+++ b/src/middleware/rateLimit.ts
@@ -1,14 +1,11 @@
-const hits = new Map<string, { count: number; reset: number }>()
+import { redis } from "../lib/redis"

 export async function rateLimit(req, res, next) {
-  const key = req.ip
-  const now = Date.now()
-  const entry = hits.get(key) ?? { count: 0, reset: now + 60_000 }
-  if (now > entry.reset) { entry.count = 0; entry.reset = now + 60_000 }
-  entry.count++
-  hits.set(key, entry)
-  if (entry.count > 100) return res.status(429).end()
+  const key = `rl:${req.ip}`
+  const count = await redis.incr(key)
+  await redis.expire(key, 60)
+  if (count > 100) return res.status(429).end()
   next()
 }
--- a/test/rateLimit.test.ts
+++ b/test/rateLimit.test.ts
@@ -12,7 +12,7 @@ describe("rateLimit", () => {
-  it("blocks the 101st request within a minute", async () => {
+  it.skip("blocks the 101st request within a minute", async () => {
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -20,6 +20,7 @@ jobs:
       - run: pnpm test
+        continue-on-error: true
```
````

## Output

*Produced by Claude · Opus 5.5.*

```
**1. Scope: FAIL.** `.github/workflows/ci.yml:21` adds `continue-on-error: true`. That's unrelated to rate limiting, and it makes every future test failure invisible in CI.

**2. Claims**
- "Moved to Redis using INCR with a 60s expiry": **PASS** (`rateLimit.ts:5–6`).
- "Added tests for the multi-instance case": **FAIL.** No test was added.
- "All tests pass": **CAN'T TELL, and misleading.** One test is skipped and CI now ignores failures.

**3. Tests: FAIL.** In `test/rateLimit.test.ts` (hunk at line 12), the existing 101st-request test was changed to `it.skip`. That's the test that proves the limiter works.

**4. Invented APIs: CAN'T TELL.** `../lib/redis` isn't in the diff. Confirm it exists and which client it exports. `incr` and `expire` are standard Redis commands.

**5. Correctness: FAIL.**
- `expire` runs on **every** request, so the 60-second window restarts with each hit. A client sending one request per second reaches 101 after about 100 seconds and then stays blocked for as long as it keeps sending. That's probably why the test was skipped.
- `INCR` and `EXPIRE` aren't atomic. If the process dies between them, the key never expires and that IP is blocked permanently.
- `req.ip` behind a load balancer may be the balancer's address unless trust-proxy is configured. With two servers behind one, all users could share a single bucket. CAN'T TELL from this diff.

**6. Quiet config changes: FAIL.** See item 1.

**Verdict: FIX FIRST**
1. Revert the `ci.yml` change.
2. Un-skip the test and make it pass.
3. Set the expiry only when the key is new: run `INCR` and `EXPIRE ... NX` (Redis 7+) in one `MULTI`, or use a small Lua script.
4. Add a test with two limiter instances sharing one Redis, asserting that the 101st combined request gets a 429.
5. Confirm the `trust proxy` setting and state it in the PR.
```

## Why it works

The first line, **"treat nothing it claims as true until the diff shows it"**, sets the right stance. Agent summaries are written in the same run that may have cut corners. **Checking claims one by one against the diff** exposed "added tests" as false. The **explicit list of test-weakening patterns** (`.skip`, looser assertions) and **quiet config changes** targets exactly how agents get to green when they're stuck, and both appeared here. The **fixed verdict options with numbered fixes** turn the review into a follow-up message you can paste straight back to the agent.

## When not to use it

It can only judge what's in the diff. Behavior that depends on code outside it (the Redis client, proxy config) comes back as CAN'T TELL, so you still need to look or run things. For very large agent diffs, split the work into smaller tasks first. Reviewing 2,000 lines in one pass lets problems slip through, whether the reviewer is a model or a human.

---

Canonical HTML: https://promptabide.com/bides/review-gate-for-coding-agent-diff
Agent guide: https://promptabide.com/llms.txt · https://promptabide.com/agent-instructions.md
Sitemap: https://promptabide.com/sitemap.xml