Sourcegraph tests a bot that routes pull requests for review

A review bottleneck shaped the experiment

Sourcegraph says coding agents have made writing changes faster, shifting more of the bottleneck toward understanding their effects and deciding whether they should ship. Its GitHub setup required an approval before merging, so routine changes and changes requiring detailed discussion passed through the same approval step.

The company already had a stamp group for small pull requests. A request such as “Can someone stamp this?” could be handled by an available teammate, but each request still required finding someone and asking them to switch context. Sourcegraph wanted to reduce that routine approval chasing while making its different review expectations explicit.

How Ship, Show, and Ask work

The bot evaluates a pull request diff, relevant code, and the author's previous contributions. It returns findings with supporting evidence, while application code applies Sourcegraph's rules and posts the selected review path on the pull request.

• Ship: no additional review is needed under the policy.
• Show: someone should see the change, but review can happen after merge.
• Ask: someone must review the change before it merges.

Sourcegraph began with recommendations only, leaving human review unchanged. The team used feedback to refine the rules and the evidence needed to establish familiarity, then rechecked affected recommendations before enabling approvals for eligible changes.

The workflow runs in GitHub Actions. The current policy looks for two merged pull requests reviewed by another person, plus a relevant contribution of code, tests, or technical documentation in the past 30 days. However, changes to authentication checks or to how sensitive data is exposed require review regardless of the author's experience.

Sourcegraph is still measuring the impact

Sourcegraph says the experiment has produced fewer requests for a quick stamp, but it is still an internal test. The company wants to understand whether Show changes receive useful attention after merge and whether colleagues remain informed after the author and reviewer have moved on.

Feedback also showed that review can catch a problem, share context, or check what the company tells customers. The team found that contribution history can provide evidence of familiarity, but experience in one part of a system does not necessarily indicate familiarity with all of it. The bot's reasoning gives people assumptions they can challenge, helping them decide whether a rule or an assessment should change.

Sourcegraph also says bot comments and repeated assessments can become interruptions, so deciding when it should speak, update, or stay quiet is part of the workflow design. The company plans a follow-up post covering reviews after merge and the bot's effect on interruptions. When human review is needed, Sourcegraph uses its Compare page to inspect the diff and navigate surrounding code.