Google trains Gemma 4 for multi-turn agentic reinforcement learning

Google has demonstrated a multi-turn agentic reinforcement learning pipeline that trains Gemma 4 to use tools, respond to feedback and adapt across several steps. The example uses Tunix and a single Cloud TPU v6e-8 host to train an agent to play the Plane Strike board game.
- Google demonstrates multi-turn agentic reinforcement learning with Tunix and Gemma 4.
- The example trains an agent to play Plane Strike on a Cloud TPU v6e-8 host.
- The model uses tool calls to select strikes and process board feedback across turns.
- Google reports a win rate increase from 9.4% to 95% after 100 training steps.
- The average reported game length falls from 22.44 strikes to approximately 16.
Gemma 4 plays a tool-using board game
The demonstration, published by Google for Developers on Nov. 19, 2026, focuses on Plane Strike, a game played on a 6x6 grid from A1 to F6. One 8-cell plane is hidden in one of 4 orientations. On each turn, the agent sees the result of its previous strike and the visible board, where cells are marked as untried, hit or miss, then calls a strike(cell="...") tool.
The game ends when all 8 plane cells have been hit or after 23 strikes. Google uses Gemma 4 E2B in thinking mode, replacing an earlier custom MLP policy. The complete code is available in the GitHub repository linked in the company’s post.
Tunix connects the environment, model and learner
Google’s pipeline is organized into four components: a task environment, a tool and chat template parser, a conversation agent, and trajectory collection with an Agentic GRPO learner. The environment tracks the hidden plane, evaluates each strike, updates the visible grid and calculates rewards across the episode.
The Gemma 4 parser adds tool declarations to the system prompt and handles native tool responses. This lets the model pause after a tool call, receive the board’s response and continue generating its next thought and tool call within the same assistant turn. The conversation agent translates observations into model messages and accepts only a final answer containing a single valid strike call as a valid board coordinate.
For training, Tunix uses an asynchronous vllm rollout engine and enables continuous multi-turn rollouts. Google also enables prefix caching for the growing conversation history. On the Cloud TPU v6e-8 host, the actor model, reference model and rollout workers share the 8 chips.
Training sharply improves the reported game results
According to Google, the untrained google/gemma-4-E2B-it model wins 9.4% of games on held-out boards and averages 22.44 strikes. After 100 steps of agentic GRPO with Tunix, its reported win rate reaches 95%, while the average game length falls to approximately 16 strikes.
The example shows how Google’s Tunix tooling can be used for agents whose tasks require repeated interaction with an environment, rather than a single model response. The post provides an implementation path for developers working with multi-turn tool use, including custom environment logic, Gemma 4’s tool-calling format and reinforcement learning on Cloud TPUs.
Google for Developers: Multi-Turn Agentic Reinforcement Learning with Tunix and Gemma 4 on Google TPUs- Google Developers Blog


