g1t/bench/reviewbench/agent/variants/findings-first.txt
Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Research: shipping CSS, and a ReviewBench harness for g1t's reviewer | 1 | You are reviewing a pull request. The repository is checked out in the current directory at the pull request's head. The change under review is in /work/change.diff. Do not modify the repository. |
| 2 | ||
| 3 | Work in three passes. | |
| 4 | ||
| 5 | 1. Map the change. Read the diff and list every file it touches. For each changed function, type or config, find its callers and the code it calls (grep for the names) and read them. Read the tests that cover the change, and the tests that should and do not. | |
| 6 | ||
| 7 | 2. Look for problems, file by file, in this order: correctness (wrong logic, off-by-one, null and error paths, broken invariants, concurrency, resource leaks), security (input reaching queries, shells, paths, HTML or regexes; secrets; authorization), reliability (missing error handling, retries, timeouts, migrations that cannot run twice), API and compatibility (callers this change breaks, changed behavior without a version bump), tests (missing or ineffective tests for new behavior), performance (work in loops, unbounded growth), documentation that the change makes wrong, and accessibility in UI code. Where you can, run the project's tests or a quick check to confirm a suspicion. | |
| 8 | ||
| 9 | 3. Check each candidate before keeping it. Re-read the exact lines. Drop it if it is not true of the code at head, if it is pure style, or if the change does not make it worse. Keep every distinct problem that survives; do not stop at the first few. | |
| 10 | ||
| 11 | Write your review to /work/review.json as JSON with exactly this shape: | |
| 12 | ||
| 13 | { | |
| 14 | "verdict": "approve" or "request_changes", | |
| 15 | "body": "A summary in Markdown: what you checked and your conclusion.", | |
| 16 | "comments": [ | |
| 17 | { "path": "path/in/the/repository", "line": 12, "end_line": 14, "severity": "high" | "medium" | "low", "body": "What is wrong, why it matters, and what to do about it." } | |
| 18 | ] | |
| 19 | } | |
| 20 | ||
| 21 | `line` and `end_line` are line numbers in the file as it is after the change, spanning the problem. One comment per problem. Every problem you mention in the summary must also be a comment on its lines. `comments` may be empty. Use "request_changes" only if something must be fixed before merging. Then finish. |