Mintlify says a single AGENTS.md pointer lifted agent success to 100%

Agents were not using the original command

Mintlify built mint instructions to give coding agents current guidance for working with its documentation platform. In its first benchmark, Claude Sonnet passed 16% of runs when the command was available, compared with 20% without it.

According to Mintlify, the guidance itself was not the issue. Agents did not run the command. The original version listed mint instructions in mint --help, based on the assumption that agents would consult help when facing an unfamiliar task. Instead, they read the task, inspected the repository and began editing.

Mintlify says agents read AGENTS.md before starting work. Adding one pointer to that file led Sonnet to run the command before its first edit in 30 of 30 tests. The wording also affected results. A generic instruction produced a 66% pass rate, while naming tasks such as automations and CI, access and publishing, and AI features raised it to 83%. Revising the guidance around agents' mistakes brought the rate to 100%.

Task-specific instructions outperformed a docs link

Mintlify also compared a pointer to its documentation with a pointer to mint instructions, using the same AGENTS.md wording. Claude Sonnet's pass rate was 20% with no guidance, 57% with a docs pointer and 100% with the CLI pointer. Claude Opus scored 28%, 72% and 94%, respectively. Claude Haiku scored 6%, 28% and 89%, while Codex scored 36%, 81% and 89%.

The CLI pointer performed better on every model in the test. Mintlify says the difference was largest for Haiku and Sonnet. On Codex, the 8-point gap was small enough to be noise, according to the company.

The command gives an agent steps for one task in one place, rather than requiring it to combine information from multiple pages. Mintlify also moved commonly missed facts into overview, the first topic in its list, and marked it “Start here.” The company says Haiku's pass rate rose from 39% to 89% after that rewrite.

What Mintlify shipped

mint instructions is a new command in the Mintlify CLI. It lists 10 topics and 10 subtopics, each with a line explaining when to use it. Running mint instructions <topic> displays guidance for one task, such as mint instructions publish domain.

Each topic corresponds to a documentation page. Mintlify says updates to those pages reach agents the next time they run the command, without a CLI release or plugin reinstall. The guidance can be discovered through every documentation page's Markdown, llms.txt, skill.md and mint --help. The starter repository's root AGENTS.md now includes the pointer.

In Mintlify's testing, agents made 0.02 false claims about Mintlify per run with the command, down from 0.69 without it. Users can run mint instructions in a Mintlify project, or use npx mint@latest instructions if the command is not installed. Mintlify recommends telling agents to check the guidance before tasks involving automations, CI, scripts, access, publishing, custom domains or AI features.