Qodo Benchmark: Claude Opus 5.5 Leads Code Review, Open-Weight GLM 5.3 Close Behind

How Qodo built the test

According to Qodo, most code review benchmarks rely on synthetic bugs or hand-labeled issues. Its own benchmark starts from bugs that actually happened. The company takes bugs that were later found and fixed, traces each one to the pull request that introduced it, and asks the reviewer to catch it in that original PR. The dataset covers 125 pull requests containing 703 known bugs.

To isolate the model, Qodo holds everything else fixed: a single reviewer, standard review effort, and the explorer subagent turned off. Findings are scored against the known bugs by an LLM judge, Claude Sonnet 4.6. Qodo reports precision, recall, F1 (its headline quality score), recall on backtracked bugs, cost per PR and median review time.

The ranking

Four models sit within 4.7 F1 points of each other: Claude Opus 5.5 (56.8%), Claude Sonnet 5.5, Claude Fable 5.1 (53.5%) and GLM 5.3 (52.1%). A second group between 46% and 50% includes DeepSeek V4 Pro, Kimi K3, GPT-6 Sol, Gemini 3.8 Flash and GPT-6 Astra. At the bottom of the table are Claude Haiku 4.5 (30.4%), Laguna S 2.1 (27.3%) and Mistral Medium 3.5 (24.0%).

Qodo says rankings can reorder from one month to the next, which is why it reruns the benchmark with every major release. Results reflect runs completed between September and October 2026, and some configurations were run once while others were averaged across repeated passes.

Reasoning level and cost matter as much as the model

Raising the reasoning level usually increases recall and lowers precision, according to Qodo, and the net effect on F1 depends on the model.

• Claude Opus 5.5 scores 56.8% F1 at high and 54.0% at xhigh. At xhigh it catches the most backtracked bugs of any configuration (55.7%), but precision drops to 54.4%.
• GPT-6 Sol scores 48.7% at medium and 48.4% at xhigh. Xhigh costs almost three times as much ($0.51 vs. $0.18 per PR) and takes more than twice as long (224s vs. 101s).
• Claude Fable 5.1 scores 53.4% at low and 53.5% at high, at a similar cost per PR.

Cost varies by more than 100x, Qodo says. GPT-6 Luna at xhigh reaches 41.7% F1 for about two cents per PR. Gemini 3.8 Flash at high reaches 47.0% F1 for $3.63 per PR, with a median review time of almost 14 minutes. Claude Sonnet 5.5 at high reaches 51.7% F1 for $0.31 per PR in 90 seconds, which Qodo says is close to the top tier at a third of the cost of Claude Opus 5.5. At xhigh, Sonnet scores 54.9% F1 at $0.73 per PR.

Open weights and different failure modes

GLM 5.3 reaches 52.1% F1 at $0.76 per PR. DeepSeek V4 Pro (49.9%) and Kimi K3 (49.0%) also finish ahead of GPT-6 Astra and Gemini 3.8 Flash. For teams that need open weights for data residency, cost or control, Qodo says the quality gap to the frontier is now a few points of F1.

Models with similar F1 can behave very differently. Gemini 3.8 Flash at high has the highest precision in the table (71.0%) but catches only 35.1% of known bugs. GPT-6 Sol at xhigh is more balanced, with 48.4% precision and 49.3% recall. Qodo argues that a team wanting few, high-confidence comments and a team wanting maximum coverage should not run the same configuration.

What it means for teams

Qodo says the best-performing combination of model, reasoning level and harness configuration is what runs in its product, and production changes when a new release shifts the ranking. The company says it is not tied to one provider, runs a mix by default, and can switch to a specific provider or open-weight model for compliance, data residency or contract reasons. It also says it can select a model and reasoning level for each review.

For organizations with approved LLM providers, Qodo positions the benchmark as an index for code review, showing what a current stack delivers and where another model or reasoning level could catch more bugs or cut cost. The company notes that the benchmark uses its own internal pull requests, that cost excludes evaluator cost and failed attempts, and that the best choice depends on whether a team prioritizes coverage, precision, cost or speed.