Cloudflare Will Score Verified Bots on Whether They Respect Site Rules

From identity to ongoing behavior

Cloudflare argues that verifying who runs a bot is not enough. It compares this to a driver's license, which shows who someone is and that they passed a driving test, but not whether they will drive responsibly later. A known bot can still ignore website preferences, overwhelm origins or evade controls, the company says, and the cause is often a misconfiguration or a bug rather than malice.

To be a Cloudflare Verified Bot, an operator broadly needs to meet three criteria: identify itself accurately and securely, do only what it says it will do, and not engage in abusive behavior. Bots use Web Bot Auth to sign requests that establish their identity, and BotBase catalogues that information in an open format. The new system focuses on the abuse criterion.

Cloudflare also separates risk from trust. Risk is the likelihood that a particular request or point-in-time action is harmful, and it can change from moment to moment. Trust is built through repeated and consistent behavior. As an example, the company says it de-listed Perplexity's bot from its Verified bot list in 2025 after it sought to access resources on websites that had opted out of scraping and then tried to circumvent controls when blocked.

What the system measures

The system measures how often a crawler respects site preferences. The output is a trust band, described by Cloudflare as a range showing how consistently a bot respects preferences and which observed behaviors informed the assessment. The company stresses it is not a permanent judgment about a bot or its operator. The initial signals are whether the bot respects instructions and whether it responds to direct feedback.

• robots.txt: The system checks whether a bot's observed traffic matches the published preferences over time, in line with RFC 9309. Cloudflare says isolated violations may not be significant, and it looks for repeated patterns that disregard site preferences. It also notes that robots.txt sets minimum rules and that anything not stated is not automatically allowed.
• HTTP 429: A 429 ("Too Many Requests") response tells a visitor to reduce its request rate, sometimes with guidance on how long to wait. The system checks whether a bot slows down, pauses or keeps sending requests at the same pace. Consistently slowing or pausing raises trust, while repeatedly ignoring the signal lowers it.

Cloudflare says bots can stay entirely within allowed paths and still place substantial load on websites, which is why the 429 signal is part of the assessment.

An accountability loop and what comes next

Cloudflare describes the system, together with Bot Preference Sync and Bot Management, as an accountability loop for automated traffic. Bot Preference Sync helps website owners publish their preferences. Observing how bots treat those preferences shows whether Verified bots honor them over time. Bot Management lets site owners decide how that traffic is handled.

The company says it is already using the system internally to evaluate how consistently Verified bots honor website preferences. It is using this initial phase to build a history of observations and refine how they become useful guidance. In the future, it plans to share the scores publicly on Radar and to expand the ways it measures behavior.

The next planned step is to make the system available to all Cloudflare customers, who would then be able to choose the trust threshold used to decide how Verified bot traffic is handled. No date for that rollout is given in the announcement.

For website owners, the approach is meant to add confidence that bots they choose to allow are respecting their preferences. For bot operators, it offers a way to demonstrate they deserve trust. Cloudflare says its goal is to keep website preferences meaningful and ensure trust is earned through sustained behavior. Customers can already use Bot Management and Bot Preference Sync, and the Verified bot population can be explored on Radar.