Sierra's τ-Entity benchmark shows voice agents still stumble on names and emails

What 𝜏-Entity measures

Sierra says it kept running into the same bottleneck in its earlier 𝜏-Voice benchmark: an agent would get a detail wrong and fail authentication before it could help the caller. 𝜏-Entity isolates that problem. It covers 200 tasks across ten entity types, including names, phone numbers, addresses and emails.

Each call focuses on a single field, and the caller is cooperative and spells accurately when asked. Most calls take less than two minutes. Each task begins with a missing value, which the agent must request from the caller and submit. Success requires the stored value to exactly match the expected one.

Sierra varies the difficulty of the value, caller behaviors such as corrections and restarts, and the audio environment. It then reports Pass³, which checks whether an agent completes the same task under regular, noise-heavy and speech-heavy conditions. This separates two questions: can the agent capture the value, and can it do so reliably as conditions change.

How current agents score

On the leaderboard, which pools all six 𝜏-i18n languages, GPT Live 1 leads. It captures the value correctly in 60.3% of regular calls, but succeeds under all three conditions on only 34.0% of tasks. Gemini 3.8 Live and Grok Voice Think Fast 2.0 reach 43.8% and 44.2% in regular conditions, and fall to 19.5% and 24.0% across all three.

English alone is easier. GPT Live 1 reaches 74.0% on regular calls and 47.5% across all three conditions. Spanish, Portuguese and Hindi stay close to English, and Gemini and Grok score above their English results there. Korean Pass³ falls to 3.5, 8.5% and Mandarin to 12.5, 24.5% across agents.

Verification helps, but repair is weak

Sierra also points to the 𝜏-Elicitation paper, which measured how often agents read values back or ask for spelling. Agents verified hard and unfamiliar entities more carefully. But only Grok verified more under noise, and none spent more effort verifying the caller voice it struggled with most.

When agents chose their own verification strategy, only 14, 41% of tasks succeeded across all three conditions. A prescribed sequence (ask, have the caller spell or read back, confirm and correct, then submit) raised that to 37, 54%, a gain of 14, 31 percentage points. The cost is time: calls take 21, 28 seconds longer. The test ran 4,800 calls in total.

Recovery is another weak point. Only 24, 37% of initially incorrect captures that agents verify are ultimately corrected. Sometimes the agent repeats the same mistake, and other times it replaces the value with a new wrong one.

Errors also add up across fields. In the multi-field study, only 53% of three-field calls succeed even though individual fields remain 75% accurate. Validating each field and allowing retries raised success on those tasks from 64% to 82%, though Sierra says it cannot isolate how much each part contributed.

What it means for voice agent builders

Sierra concludes that reliable voice agents need to recognize uncertainty, verify the right things and repair mistakes. It also cautions that not every interaction needs the longest verification process, because the extra care costs time. The challenge, the company says, is knowing when that effort is needed.

𝜏-Entity launches alongside 𝜏-i18n and covers the same six languages. Sierra credits Victor Barres at Mercor for the collaboration. For teams building voice agents, the benchmark offers controlled tasks for tracking progress across entity types, caller behaviors and environments.