OpenRouter Launches Benchmarks to Compare AI Model Routers on Quality, Speed and Cost

What the new benchmarks page does

OpenRouter says the page is meant to help users compare performance across the growing number of router options. Every router was run through a set of benchmarks from varied domains and scored on quality, speed and cost. Individual models that represent both best-in-class cost efficiency and intelligence appear next to the routers for comparison.

The company says it will keep adding new routers and updating scores as the routers improve. OpenRouter introduced two of the first model routers, the Auto Router and the Free Models Router, which it says were built to give users a dependable option that stays current with the latest models at their preferred level of cost.

The Router Index combines the three dimensions into one 0 to 10 score for each benchmark. By default it weights quality (the benchmark score) at 60%, time per task at 20% and cost at 20%. Users can drag a slider on the page to change those weights.

Why routers exist, and where they struggle

OpenRouter points to three reasons for routing. The first is cost: it says GPT-6 Astra's average price paid per token is over 48x higher than that of DeepSeek v4 Flash, and that running an average 10 to 49 turn session in Codex is 21x more expensive. Routers can hand parts of a task to a cheaper model.

The second is task diversity. The model that performs best on legal research may not be the best for coding, so a router can switch between them. The third is session stage: different parts of an agentic session need different levels of intelligence, and routers can change models or reasoning effort for complex planning versus mundane execution.

The company also lists the downsides, which it says can sometimes make a router perform worse than a single model:

• Switching models makes requests more expensive while the input cache is rebuilt.
• Routers often lack enough information to judge task complexity from the prompt alone.
• Heuristic signals for session stage may not match the model's next step.
• More processing on routing decisions adds latency.

The benchmarks are intended to show which routers overcome these challenges.

Types of routers that were tested

OpenRouter notes that the word router covers many different things. Provider routing, which it says it has done since day one, picks among inference providers based on price, speed, uptime and data policy, and falls back to another if one fails. Model routing sends a request to a router that decides which model answers.

The benchmarked model routers fall into three groups:

• Blends of models: Unbiased's Pareto and Sakana's Fugu are inference providers that use a blend of models, sometimes with frontier escalation. They do not disclose which models are used or how often switching happens, and they bill at standardized per-token rates.
• Per-turn selection: Auto Router and Jev Router choose a model for each turn. The selection is transparent, and billing follows the standard rate of the chosen model.
• Pre-defined model sets: NVIDIA's Switchyard pairs a cheaper model with a stronger one and switches between them as an agent works. Several pairs were benchmarked, but any pairing can be used.

Pareto Code, the -latest model slugs and Fusion were not benchmarked. OpenRouter says they serve specialized purposes that call for different comparison methods. Fusion runs the same request across a council of models and synthesizes the best answer.

What it means for users

OpenRouter cautions that, like any benchmark, the results represent general tasks rather than a user's own work. All the listed routers are available on OpenRouter, and users can swap them in for a model in any app, harness or through the API. The page is designed to help people find the router and model combination that fits the performance characteristics they need, and to decide when, and whether, a router is worth using.