OpenRouter rebuilds provider routing to reward speed and reliability, not only price

Why price-only routing fell short

According to OpenRouter, about 77% of requests on its platform go to models served by more than one provider. When a request carries no explicit sort or provider list, the router has to pick one. Until now it did so almost entirely on input token price: the cheaper a provider, the larger its share of traffic, and failed requests moved to the next cheapest option.

The company says this delivered the lowest prices in the ecosystem but caused problems. Cheap providers won even when their service was poor, and users waited through fallbacks. Rankings were static, so a provider had to degrade badly before traffic shifted away. Providers could only compete by cutting prices, which put pressure on their investment in reliability. The system could also be gamed: a provider could list an artificially low input price and recover the difference through overpriced cached or completion tokens.

How the Routing Cost works

The new algorithm continuously monitors each provider and computes a real-time Routing Cost. It covers the actual price, the cost of waiting for a response, and the cost of retries when a provider fails or returns an unusable answer. OpenRouter frames it as two questions: what does one try cost, and how often does it work?

• Effective price: calculated from recent measured traffic, including the provider's real cache hit rate and the model's average mix of input and output tokens. It can differ a lot from the list price.
• Throughput: measured as output tokens per second end-to-end. A faster provider can charge more and still win a request. The wait is converted into cost using a rate set per model.
• Time to first token: counted separately because interactive use, such as agent sessions with many consecutive tool calls, depends on it most.
• Rate limits: HTTP 429 responses add the wait plus a penalty, weighted by how often a provider rate limits.
• Reliability and quality: a failed try still costs its price and wait, so the router calculates the expected cost of retrying until one works. If 1 in 5 tries fails, you pay for 1.25 tries per good answer.

On quality, the main check today is whether tool calls come back well formed, and OpenRouter says more checks are being added. A provider flagged for broken tool calls, or for benchmark accuracy well below what the model should score, drops far behind and receives only a small amount of test traffic until the flag clears.

Sharing traffic and guardrails

Sending everything to the single best provider would overload it, so traffic is shared. Providers inside a competitive edge share most of the traffic, and being slightly behind the best costs only a little. Beyond that edge, share drops off fast.

Three guardrails apply. A floor gives every provider a trickle of traffic so improvements can be noticed, and new endpoints start from the model's typical numbers. A back-off reduces traffic to providers that start failing or rate limiting. A cap reflects how much load each provider can take before returning 429 errors, with the rest spilling to others. According to OpenRouter, adding capacity raises the cap automatically.

Reported results and what stays the same

OpenRouter reports improvements in TTFT (16.9%), throughput (16.3%) and capacity errors (a 67% reduction), with only minor increases in cost (6.0%). It says this comes before providers have had time to optimize for the new algorithm.

Some behavior is unchanged. Requests that set provider.order bypass the new logic and follow the specified order, with price-based as the default. Setting provider.only or provider.ignore applies the new algorithm to the remaining providers if more than one is left. An explicit provider.sort of price, throughput or latency uses that sort instead. Only the first request in a conversation is algorithmically routed, and the conversation stays pinned to that provider to keep the prompt cache warm, falling back automatically if the provider becomes unavailable. A failed try is retried with the next highest ranked provider.

For users, the company says no code changes are needed, and price-only routing remains available through pinning and sorting. For providers, the incentive shifts from undercutting on price to delivering faster, more reliable inference.