Google DeepMind unveils Genie 4 world model with native audio and dialogue

A world model that builds on Gemini

DeepMind describes Genie 4 as natively multimodal. Ambient noises, music and interactive dialogue are generated at the same time as the simulated world, rather than added afterwards. The company says Genie 4 is its first real-time world model to generate audio natively, and that every world carries the ambient sound of its own setting, from wind in the trees to waves crashing at the beach.

The model draws on the Gemini family. According to DeepMind, Gemini Omni anchors the simulations in real-world appearance and behavior, while other Gemini models provide the reasoning needed to keep a simulation coherent and continuous. The idea, the company says, is that you can set an event in motion and Genie 4 simulates what happens next: how the world responds, how it sounds, and how the people in it move, react and speak.

Four capabilities: steering, physics, people and sound

DeepMind says Genie 4 advances four key capabilities. The demonstration recordings show users navigating with standard keyboard controls for movement and camera, and using text prompts to steer the scene as it evolves.

• Real-time shaping: Users can change the weather, move the camera, or introduce a new person or object in natural language. Gemini adapts the prompt to fit the surrounding scene and, between inputs, monitors and steers the world so it keeps progressing cohesively.
• Physics and human action: DeepMind says the model is built on an advanced physical understanding of the world and simulates scenarios such as aerodynamics and collisions. It calls the simulation of people the most dramatic leap, with characters that move naturally, respond organically to their environments and can be steered in real time.
• Sound: Beyond ambient audio, Genie 4 introduces interactive dialogue that adapts on the fly. Characters converse and react to user inputs or events in the world.

The example prompts published by DeepMind range from a saxophone player on a floating platform in a misty lake to a crowded 1980s Japanese train station, a macro shot of a falling maple seed and a stained-glass landscape that can be explored in three dimensions.

Availability and intended uses

Genie 4 is being rolled out as a research preview for trusted testers. DeepMind says it is gathering feedback before releasing the model to consumers and developers, starting with Google AI Ultra subscribers in Project Genie and early access partners of the Gemini API and Google AI Studio. A waitlist for Genie 4 in Project Genie is open. The source gives no pricing or general release date.

DeepMind frames the work as part of its broader goal. "Understanding and simulating the world is a crucial aspect of building AGI," the company writes in its announcement. It adds that Genie 4 advances its ability to create safe, infinitely variable simulations that can help with everything from developing training environments for robots and agents to unlocking the creation of new interactive experiences.

For now, access is limited to trusted testers, so the claims about realism, physics and dialogue come from DeepMind's own demonstrations. Wider availability through Project Genie, the Gemini API and Google AI Studio will show how the model performs outside the company's examples.