Inworld launches inference service built for consumer AI apps

What Inworld is offering

Inworld Inference does not list thousands of models. Instead, the company says it goes deep on the open models that consumer apps choose, including Gemma 4, GLM, DeepSeek and MiMo. Each model gets its own serving build, according to Inworld.

The techniques the company names include new attention kernels, realtime decoding in which a small draft model writes ahead and the full model checks it in one pass, lower-precision weights "wherever the evals hold", and a router that sends every turn back to the server that already holds its context. Inworld says every build ships only if it matches the eval scores of the build it replaces.

Developers can use the hosted models with no rate limits and a service tier per request, route across models with automatic failover, or bring their own model for a dedicated deployment.

Why consumer AI is a different workload

Inworld argues that consumer apps differ from enterprise and coding use on almost every count. In conversation, people leave 0 to 300 ms between turns, so the first token has to arrive well inside that window. The company says the median first token on Gemma 4 26B now lands in 117 ms, which leaves most of the window for voice synthesis and network time.

Prompts are also long relative to replies. Every turn resends the whole history, and across the apps Inworld served in the 28 days to 23 September, median prompts were 13 to 36 times the length of replies. One interaction can also fan out into many model calls, such as moderation, memory, ranking and the reply itself. Because users pay $5 to $20 a month or nothing, Inworld says each call has to cost almost nothing, and it aims to bring costs to a few cents per user per day.

Inworld also points to market size: it cites a16z with PNC data showing that only 2% of US households and 3% of US consumers spend anything on AI.

The numbers Inworld reports

All figures below come from Inworld and are based on its own router measurements and OpenRouter data, which the company describes as snapshots rather than a matched workload benchmark.

• Gemma 4 26B speed: a median first token of 0.17 s and a p95 of 0.31 s, against a median provider's 0.61 s and 2.42 s. That is 3.6 times faster at p50 and 7.8 times faster at p95.
• Gemma 4 26B price: $0.07 per 1M input tokens, against $0.1115 for the median of the 12 OpenRouter providers that do not train on user data, at list prices on 12 October 2026.
• MiMo 2.6 Pro: the lowest input price among providers on OpenRouter, with a median first token of 0.39 s and a p95 of 0.91 s. Inworld says that is 2.6 times faster at the median and 4.6 times faster at p95 than the next-fastest provider.
• Reliability: on GLM 5.3 Flash, 99.999% of more than 20 million requests in 30 days completed without an error. Inworld counted 98 failures, including server errors, timeouts and capacity rejections.
• Efficiency: the same hardware serves 1.8 times the requests it did in September, with a faster median first token.
• Caching: up to 90% of each prompt is served from cache, and cached input costs up to 99% less.

Inworld says its input prices are the lowest of any provider that does not train on user data on three of the five models it features.

Scale and what it means for developers

Inworld says almost every token it serves comes from consumer apps, tens of trillions of tokens a month, 25 times what it served in March. On the most-used Gemma 4 models, it says it processes up to 1.8 times the daily tokens that OpenRouter publicly reports across all its providers combined, based on data from 4 to 10 October 2026.

For developers building voice companions, tutors or coaching apps, the pitch is that speed, price and uptime are tied together. Slow replies push users back to buttons, and a ten-minute outage can send them to a competitor. Whether the figures hold for a given app will depend on its own traffic, since Inworld itself notes the comparisons are not a matched benchmark.