CodeRabbit says Codex helped it ship 20 times more software

Code review becomes a control layer for AI-generated code

CodeRabbit provides an AI-powered governance and control platform for reviewing, understanding, and securing code produced by AI agents. According to the company, its tools are used by more than 20,000 organizations and 150,000+ open-source projects.

Harjot Gill, CodeRabbit’s co-founder and CEO, says code review becomes a key constraint when companies adopt coding agents. “The first bottleneck that every company hits as soon as you adopt coding agents is code reviews,” Gill says in the OpenAI account.

CodeRabbit evaluates new models against its own benchmark of known issues. Gill estimated that GPT-6 Astra caught 50% of the issues in that dataset at the time of the interview, placing it at the top compared with other models in that benchmark. The company says the benchmark changes as agents create different mistakes and as later fixes expose issues that earlier reviews missed.

More subagents as model costs improve

CodeRabbit built its first product with GPT-3.5 Turbo and later found GPT-4 effective for code review, according to Gill. The company then adopted OpenAI o1 and OpenAI o3 early. Because its reviews run asynchronously in the cloud, CodeRabbit could use models that took longer to reason through problems in exchange for more capable analysis.

Gill says improvements in model pricing and caching have allowed CodeRabbit to add more AI agents and move capabilities from research into its product. The company initially used one or two subagents for code navigation. It now uses lower-cost models such as GPT-6 Luna to support more subagents, while GPT-6 Astra handles planning and orchestration.

“Now we are using agents everywhere on the use cases we couldn't even afford in the past,” Gill says. CodeRabbit says this broader validation helps it provide more explanations to developers who need to understand and trust changes they did not write themselves.

Codex changes CodeRabbit’s software delivery process

Using Codex, Gill says CodeRabbit has been shipping roughly 20 times more software over the last 6 months without adding engineers. He personally estimated that he had pushed 2,500 to 3,000 pull requests in the 30 days before the interview.

The increase required changes beyond adopting Codex. Running multiple agents exposed constraints involving laptop memory and compute, testing, software compilation and packaging tools, and dependencies. CodeRabbit streamlined its release process and reconsidered where human approvals were needed. It also uses its own platform to automatically ship low-risk changes that do not require human intervention.

“We had to actually completely rethink the entire software delivery,” Gill says. CodeRabbit has also built a Codex-powered Slack bot that triages production incidents, Slack messages, and support tickets. The bot can return a pull request for the team to review and merge.

The company uses Codex outside engineering as well. Marketing and operational work can be managed through code, including Google Ads deployed through pull requests. A content studio repository lets Codex create slide decks and customer-facing documents. “We are running the company as code as much as possible,” Gill says.

Daybreak powers security testing and CodeRabbit Secure

CodeRabbit has run agents in sandbox environments since 2023, cloning repositories and allowing agents to write code for reviews. More recently, it used OpenAI’s Cyber model through Daybreak for internal penetration testing, probing its own platform for ways an attacker could break in.

Gill says the testing uncovered “multiple vulnerabilities, advanced hacks that we could not even imagine could happen.” CodeRabbit has since brought these capabilities to customers through CodeRabbit Secure, its latest offering. The product uses Daybreak with Codex models and security-focused agents to find and fix vulnerabilities.

During a promotional offer over the past two weeks, CodeRabbit says it covered $1M in free deep scans for users. The scans found over 43,000 security vulnerabilities across 13,000 deep scans, all powered by Codex models. The company says thousands of those vulnerabilities have already been fixed by customers.

For users, the announcement points to a model in which AI agents produce more code while CodeRabbit supplies validation, prioritization, explanations, security checks, and verification. Gill expects teams to need stronger controls as background agents respond to alerts and incidents without a person initiating every task.