Google tests cheap classifier to route Gemini tasks and cut agent costs

Cost per task as the metric that matters

According to the post, written by Sergio Villani of Google Cloud AI, per-token API prices keep falling while engineering bills for autonomous coding agents keep climbing. The author says the metric that now governs production economics is cost per successful task: total spend across every loop iteration, tool call, retry and internal thinking pass, divided by the number of tasks that pass verification.

The problem, the post says, is that agents often call a high-thinking reasoning configuration for every micro-decision, paying deep-reasoning prices for questions a lightweight configuration already answers accurately. The proposed fix is a lightweight routing layer that intercepts routine decisions before they reach an expensive model.

The example uses TypeSafe's Jev classifier placed in front of Google Antigravity's model calls. Jev is described as a specialized structured-output classifier that evaluates a Choice from a list, a numeric Score or a Noul (a 0 to 1 probability for a binary statement) in a single forward pass. Google states it has no partnership with TypeSafe and that the article is not an endorsement. The same pattern can be built with other classifiers, including Google-native ones, such as the one in the Getting started with discriminative models codelab, which runs on DiffusionGemma, Google DeepMind's open-weights model.

Why simple heuristics fell short

The first attempt used a single broad Choice question, simple or complex. On 16 coding-agent tasks, Jev sent nearly everything to simple, even tasks needing an LRU cache trace or modular exponentiation. Replacing it with one atomic Noul question fixed the separation. On those 16 tasks, Jev agreed with hand-labeled difficulty on 13 of 16 in the first run and 14 of 16 on the second pass.

The team also tested a zero-cost Python heuristic that matched words like sequence, modulo and race condition and escalated any prompt over 35 words. It scored 15 of 16, or 93.8%, on the questions used to write it. After freezing the code and adding 34 unseen held-out tasks across seven buckets, it fell 23.2 percentage points to 70.6%.

• Simple tasks such as extracting a port from a Kubernetes YAML manifest or a line number from a Python traceback exceeded 35 words and were falsely escalated.
• Short multi-step traces, such as mutable default arguments across calls or C short-circuit evaluation, lacked the hardcoded puzzle words and were sent down the simple path.

Jev classified all 34 unseen tasks in agreement, reaching 48 of 50, or 96.0%, across the full suite.

Output caps do not reduce thinking

The post warns against assuming that a low maxOutputTokens setting makes a reasoning model think less on easy tasks. Comparing gemini-3.8-flash at maxOutputTokens=300 with the same model at 4096, both spent almost identical thinking tokens on the 27 tasks Jev labeled SIMPLE. The capped version cost slightly more on easy tasks and broke 7 of the 25 hard ones: in six the model ran out of room while reasoning, and in the seventh it compressed its reasoning and answered wrongly.

A Jev router using the capped model for easy tasks and normal Flash for hard ones got all 50 right, but cost slightly more than plain Flash and was slower. The author's conclusion: if the cheap option does not make the model think less, no router can save money.

The strongest baseline and what beat it

A second round tested Flash-Lite, which does not think by default, and Gemini 3.8 Flash with thinking set to low. The strongest setup without a router was Gemini 3.8 Flash on low thinking, which solved all 50 tasks.

The Jev router sent every task to Flash-Lite first and escalated only what Jev flagged as complex. It also solved all 50 tasks. Flash-Lite alone answered every simple task in half a second but missed 6 of the 25 hard ones; Jev flagged all 6 as complex with high confidence. According to the post, the router can help reduce cost and response time versus low thinking everywhere, though on short prompts the cost difference per task is small and the shorter wait is the bigger benefit for interactive agents.

The keyword heuristic was cheaper only because it erred: it sent 5 hard tasks to Flash-Lite and 2 failed, for 48 of 50. Escalating hard tasks to Flash on default thinking cost less than default thinking everywhere, but more than low thinking everywhere. A heavier escalation target only makes sense when the low setting starts failing.

What it means for agent builders

The results point to a practical order of work: measure cost per successful task, find the strongest setup without a router, and only then test whether a classifier can beat it. The post cautions that figures come from single-run measurements on a sample suite, and that latencies, token counts and billed costs will vary with prompt length, context caching and production load. Speed comparisons against default Flash or Pro mix two test rounds and should be read as approximate. The source text is cut off while listing further limits, including one run per setup and 50 prompts with hand-assigned difficulty.