Docker: A read-only AI code reviewer invented a CVE and a fake attack

A careful setup and a boring change

The story, shared by Pratik Kumar Singh as part of Docker's Agent Hall of Shame contest, concerns a platform engineer who built a code review bot for his team. Every time someone pushes code, the bot reads the changes and writes a review into the pipeline, similar to the first pass a human teammate would give.

According to Docker, the setup was deliberately cautious. BuaJi runs on local models through Ollama, so the model weights sit on the engineer's own hardware and no request goes to an AI vendor. It runs as a job in his CI pipeline, which makes the whole setup self-hosted, and company source code never leaves the building.

He also gave the bot one firm rule: it can read and it can comment, nothing more. It cannot run terraform apply, cannot run kubectl apply, and cannot push a commit.

The change that triggered the incident was a version bump of a bundled PostgreSQL copy, from 16.11 to 16.13. Docker describes it as about as boring and responsible as a code change gets, since its only purpose is to close publicly known security holes.

What BuaJi wrote

The bot answered with roughly fifteen hundred words rejecting the change. A normal review comment, Docker notes, is a sentence or two. According to the account, the report:

• called the version bump an "architectural takeover"
• claimed the new PostgreSQL build was a vehicle for sending internal data to a third-party company, which it named, though the company does not exist
• referred by name to Kubernetes resources that never existed, although BuaJi had no access to the cluster
• wrote Go scripts to "isolate the infected PostgreSQL nodes," even though nothing was infected
• told the engineer to revoke all IAM access immediately, which would lock out everyone in the organization

The argument rested on a CVE number that has never been issued by anyone. Docker points out that a CVE identifier is among the easiest things in security to verify, because it can be looked up in a public registry in seconds. The report still looked convincing: clean Markdown, YAML that would have parsed, and an urgent, confident tone. The engineer spent about two minutes reading the CI logs, saw that none of it was real, and moved on.

Why read-only was not enough

Docker stresses that the read-only rule worked as designed. BuaJi could not carry out its own recommendations, and the episode cost the engineer two minutes. The gap, according to Docker, is that the rule controlled what the bot could do, not what it could say. The person reading the output usually has the access the agent was denied, so the bot could write scripts and hand them to someone who could run them.

Docker offers a scenario: a newer engineer on call at two in the morning, half awake, facing a long and alarming report. Revoking all IAM access in production would be an outage by itself, and running an unreviewed Go script against database nodes could be worse. In that case the person would have done the damage in good faith. No permission model would flag this, Docker says, because no permission was violated. Only the content of the report was false.

What Docker Sandboxes change, and what they do not

Docker is direct about the limits: a sandbox is not a fact-checker, and no microVM boundary will make a review comment accurate. What a boundary changes is what fabricated output can do once it exists.

In the story, the read-only rule lived in a convention, configured by one engineer in his own pipeline. Docker argues such rules can erode when someone is in a hurry, a teammate copies a config and drops a section, or a temporary exception becomes permanent.

Docker Sandboxes run agents inside isolated microVMs, each with its own kernel, its own filesystem, and a network that denies everything by default. According to Docker, an agent inside one can invent anything, but the invention has nowhere to go unless access was allowed ahead of time. The read-only rule becomes a property of the environment rather than something a person must remember, and the policy log can show that a report claiming to have inspected a cluster never could have.

That protects against the agent, Docker says, but not the human reading the output. For that, it recommends treating agent review output like agent input: untrusted until something checks it.