Qodo halves recommendation agent cost with GEPA-optimized prompt

What the recommendation agent does

Qodo argues that false positives are only one source of noise in AI code review. A finding can be technically reasonable and still be something a team will repeatedly ignore. To address this, Qodo personalizes review using a team's own history.

The recommendation agent runs after the review agent has produced its findings. It receives those findings, the pull request diff and repository context, then searches a preprocessed index of past PR suggestions and the team's responses. For each finding, it predicts whether the team is likely to act on it, rather than judging technical correctness alone.

The agent returns "yes", "no" or "maybe", along with reasoning and references to relevant past PRs. In Qodo's PR review, these predictions appear as star ratings: three stars for "yes", two for "maybe" and one for "no".

Inconsistent labels and a custom GEPA

According to Qodo, the agent's labels were not stable under the current production prompt. Running it twice on the same PR could produce different labels. On an evaluation set of 200 PRs, 20.7% of labels soft-flipped between "yes" and "maybe", or between "maybe" and "no", while 4.3% switched directly between "yes" and "no".

Qodo focused on the system instructions, keeping the same task, tools and output labels. No model weights were trained. GEPA is an algorithm that iteratively improves prompts by analyzing an agent's outputs and evaluation feedback. Because long agent traces limit how many examples a reflector can inspect together, and noisy behavior can make an accident look like a general lesson, Qodo built its own two-step version:

• Each PR is run twice, and an LLM analyzes the full traces to produce a structured assessment of what went right, what went wrong and what might explain differences between the runs.
• The structured analyses across the whole training set are aggregated in a single LLM call with high reasoning effort.

The dataset came from internal Qodo PR reviews, audited by a Fable agent with one agent per PR. It held about 400 PRs from five repositories, split chronologically into roughly 120 training, 80 validation and 200 test PRs. Each example has up to 10 findings with a boolean acceptance label.

Optimization started from a 43-word prompt describing only the agent's goal. Qodo ran a single wide step over the entire training set, limiting each reflection to 200 words and the final prompt to 1,000 words. On offline validation, hard flips fell from 8.2% to 2.2%, and total flips from about 27% to 20%, relative to the starting prompt.

Production A/B test results

Qodo compared the selected prompt with the existing production prompt in a 50/50 A/B test assigned by PR, across all its customers. Only the system prompt changed; the model, tools and UI behavior stayed fixed. The test covered 27,582 review runs across approximately 21,000 PRs.

• Among findings developers implemented, the share labeled "no" (the miss rate) fell from 7.83% to 4.46%.
• Mean recommendation agent cost per review run fell from $0.24 to $0.12.
• Mean latency fell from 40 to 30 seconds.
• Precision was comparable, with no statistically significant difference.

Qodo notes limits on these figures. The quality analysis covered only PRs with available attribution labels and at least one implemented finding. The miss rate measures missed developer action, not the rate of incorrect review suggestions. The cost and latency savings apply to the recommendation agent component of code review.

According to Qodo, the revised prompt gave the agent clearer guidance on finding relevant historical evidence, stopping redundant searches and handling uncertainty, and sharpened the distinction between predicting a team's response and judging code quality. The prompt was promoted to production.

Lessons and what it means

Qodo draws four lessons. Start with a minimal prompt, which can be easier to optimize than a manually tuned production prompt. Learn from full agent traces, including tool choices and evidence gathering, not only inputs and outputs. Measure noise by running the agent on the validation set several times before trusting scores. Run each recipe more than once and compare candidates.

For teams using Qodo, the company's reported result is that developers keep more of the findings they act on, at a lower serving cost for this component. For the wider field, the work is an example of treating prompt optimization as a measurable engineering process with explicit tradeoffs among quality, cost, latency and stability.