Gradium TTS Reaches 155 ms Time-to-First-Audio on Baseten

A TTS model built for voice agents

Gradium's text-to-speech model converts written text into spoken audio and is designed specifically for real-time AI voice agent interactions. The key metric is Time-to-First-Audio (TTFA), the time between sending a TTS request and receiving the first playable audio. Baseten describes it as critical for natural-sounding conversations.

On the Coval TTS benchmark, as captured May 4, 2026, Gradium TTS reached a median TTFA of 155 ms with an interquartile range of just 2 ms. Baseten says that spread is tight enough for developers to set reliable latency budgets across the rest of their pipeline. It also says the result puts the model in the faster-than-human-speech tier, which leaves voice agents time to reason or retrieve context without breaking conversational flow.

How Delayed Streams Modeling works

Gradium TTS is a streaming model. It can start speaking before the upstream text generator, such as an LLM, has finished producing its response. This relies on Delayed Streams Modeling (DSM), a framework developed by Kyutai. DSM treats text and audio as two parallel streams on a shared clock, with one step every 80 ms.

The audio stream runs a few steps behind the text, so the model sees what comes next, much as a human speaker adjusts tone, inflection and pacing based on upcoming words. A longer delay gives the model more context and better voice quality, while a shorter one lowers latency. According to the source, Gradium TTS starts speaking after the first few tokens rather than after the full generation.

Each 80 ms audio frame is produced by two transformers:

• Temporal transformer (the "reader"): takes in the newest text token and the previous audio frame, produces a vector describing what the next frame should sound like, and signals when it is ready for the next word. This keeps speech paced naturally when text arrives unevenly.
• Depth transformer (the "refiner"): turns that vector into an audio frame encoded as up to 32 tokens. The first token captures the most important features and each later one adds finer detail.

The larger temporal transformer runs once per step. The smaller depth transformer runs up to 32 times per step but stays cheap because it never looks back in time.

GPU optimizations for small models

Baseten explains that in large language models the GPU is the bottleneck, so the CPU can queue the next kernel launch before the current one finishes. In smaller models such as streaming TTS, GPU kernels finish faster than the CPU can queue them. The CPU then becomes the latency bottleneck and GPU utilization drops.

Gradium addresses this in two ways:

• Kernel fusion: using torch.compile and Triton, operations that previously needed several GPU kernel launches are merged into one.
• CUDA Graphs: the full kernel sequence for a forward step is recorded once, and on each later step the CPU issues a single replay command.

In Gradium's published benchmarks, enabling CUDA Graphs cut median TTFA by more than 2.5x at small batch sizes.

Handling bursty traffic with faster cold starts

Voice AI traffic tends to be bursty, with high demand at the start of business hours and during events such as outages or marketing campaigns. Provisioning for the peak leaves GPUs idle, while provisioning for the average hurts performance at the start of each spike. Autoscaling helps, but a new replica cannot serve calls until its model weights are loaded into GPU memory.

The Baseten Delivery Network (BDN) reduces cold start time in three ways:

• Push-time mirroring: weights are copied into Baseten-managed storage at deploy time, with a content-hashed manifest, so upstream is touched once.
• Three-tier caching: each GPU node checks local NVMe disk, then an in-cluster peer cache, then the mirrored origin, delivering weights to H100 nodes at over 2 GB/s.
• Coordinated downloads: each chunk is pulled from origin by one node and shared with peers, so origin bandwidth for a 50-replica scale-up is 1x the model size, not 50x.

Because weights arrive before the model container starts, the transfer also happens outside the billable GPU window. Baseten says the result is 2-3x faster cold starts.

What it means for developers

Gradium TTS is available now as a dedicated deployment on Baseten. For teams building voice agents, the pairing targets two problems: fast first audio for each interaction, and the ability to follow sudden swings in call volume without callers waiting on a cold replica. Baseten also offers a TTS Latency Race demo with a code repository for those who want to test the model themselves.