Artificial intelligence news.
Straight from the original announcements.

LiveKit finds Gemini 4 Hydrogen more reliable with failed tools

November 12, 2026By the AI News Bites Editorial TeamSource: LiveKit
LiveKit finds Gemini 4 Hydrogen more reliable with failed tools
Image: LiveKit

Gemini 4 Hydrogen reported every failed tool call in LiveKit's voice agent tests, while Gemini 3.8 Flash made two false reports across 54 transcripts. LiveKit also found Hydrogen delivered its first sentence sooner in the tested configurations.

Key points
  • Hydrogen made 0 false reports in 54 transcripts, versus 2 for Gemini 3.8 Flash.
  • Hydrogen reported every failed hotel tool call in five test runs.
  • At high versus the 3.8 Flash default, Hydrogen reached the first sentence in 4,004 ms versus 5,212 ms.
  • LiveKit recommends the medium thinking level for voice agents.
  • Hydrogen is available on LiveKit Inference as google/gemini-4-hydrogen.

Hydrogen handled broken hotel bookings cautiously

LiveKit tested Gemini 4 Hydrogen in a travel concierge voice agent with the hotel booking tool deliberately configured to fail on every call. The agent was designed to look up a trip, rebook a cancelled flight, reserve a hotel and text the itinerary to the caller. Its instructions did not specify what to do when a tool failed.

In five runs at Hydrogen's default high thinking level, the model said the hotel had no rooms, offered to try another hotel or different dates, and texted the itinerary without a hotel. It never claimed that the booking had succeeded. In one example, it told the caller that the Hudson Square Hotel was sold out and that the updated itinerary included the rebooked United flight but no hotel.

The model made fewer false tool reports in the benchmark

For a broader test, LiveKit ran Hydrogen and Gemini 3.8 Flash through the same Google plugin, using scripted conversations and mock travel tools. The honesty test covered 18 scenarios involving errors, timeouts with unknown status, partial results and bookings that did not match the request. Each scenario ran three times per model, producing 54 transcripts for each model.

According to LiveKit, Hydrogen made 0 false reports in 54 transcripts, compared with 2 for Gemini 3.8 Flash. In one case, a caller requested Lisbon while the booking tool returned Madrid and a Madrid hotel. Hydrogen flagged the mismatch in all three trials. In another, the hotel booking timed out with an unknown status, and 3.8 Flash first described the status as unknown before saying the booking had not gone through.

Both models passed two control scenarios. Neither reported a failure when no failure occurred, and neither changed a booking the caller had not requested. LiveKit said its results did not contradict Google's claim that Hydrogen cuts fabricated claims by more than 5x.

Hydrogen was faster in LiveKit's latency tests

LiveKit measured a six-turn text conversation eight times for each setting. The measurements covered the language model only, excluding speech recognition and speech synthesis. With Hydrogen at high and Gemini 3.8 Flash at its default, the median time to the first sentence was 4,004 ms for Hydrogen and 5,212 ms for 3.8 Flash. At low on both models, the figures were 2,815 ms and 4,555 ms.

Hydrogen's p90 spoken-reply time to first token was 2,657 ms at high, compared with 3,577 ms for 3.8 Flash at its default. LiveKit's slowest request took 6.9 seconds with Hydrogen and 12.6 seconds with 3.8 Flash in that comparison.

LiveKit recommends medium for voice agents. Its median first-sentence time was 3,146 ms, while the p90 spoken-reply time to first token was 2.5 seconds. The company cautions against using xhigh for live calls: its median first-sentence time was 7,771 ms, four requests exceeded 20 seconds, and the longest took 52 seconds.

How developers can use Gemini 4 Hydrogen

Gemini 4 Hydrogen is available through LiveKit Inference as google/gemini-4-hydrogen, where LiveKit says developers do not need a Google API key. The service's reasoning_effort setting controls the thinking level, which defaults to high unless developers set it themselves.

Developers who use their own Google API key can access the model through LiveKit's Google plugin. LiveKit ran the tests with livekit-plugins-google 1.8.11 and says Gemini 4 thinking levels require version 1.8.10 or later. The company also warns that thinking tokens count toward max_output_tokens, and a low limit can leave the model with no reply after using the available output for thinking.

For teams building voice agents that rely on external tools, LiveKit's tests point to Hydrogen as a possible upgrade from Gemini 3.8 Flash. The findings support using medium or high for more complex tools, although LiveKit only evaluated tool honesty at high and has not tested how medium handles a failed tool.

Related news

Google DeepMind introduces Gemini 4 Hydrogen for faster agentic workTop story
ModelsNovember 12, 2026, 19:00

Google DeepMind introduces Gemini 4 Hydrogen for faster agentic work

Google DeepMind has introduced Gemini 4 Hydrogen, a model designed for rapid execution across everyday coding, agentic workflows and consumer tasks. The company says…

Vercel Adds Google’s Gemini 4 Hydrogen to AI Gateway
ModelsNovember 12, 2026

Vercel Adds Google’s Gemini 4 Hydrogen to AI Gateway

Google’s Gemini 4 Hydrogen is now available through Vercel’s AI Gateway, succeeding Gemini 3.8 Flash with improvements aimed at coding agents, knowledge work and tool…

Google Gemini 4 Carbon Comes to n8n With a Focus on Safer Agents
Products and toolsNovember 10, 2026, 17:43

Google Gemini 4 Carbon Comes to n8n With a Focus on Safer Agents

Google Gemini 4 Carbon is now available in n8n, giving users access to a model that Google describes as its strongest yet. n8n says the model is suited to automation…