Why The Agentic Data Company Records Real Calls to Train Voice AI

Real conversations instead of scripts

According to Vestergaard, many speech datasets are built from structured recording sessions in which participants read scripts, respond to prompts or act out conversations. He says these methods produce clear recordings but often fail to capture how people communicate naturally. Real conversations, he explains, depend on relationships, intentions, context and shared experiences, and asking strangers to imitate everyday interactions can change how they speak.

The company instead collects speech during activities people already do. It builds applications around those activities, starting with a calling app similar to WhatsApp and later expanding into tools such as meeting recorders. The settings it records include phone calls between family members, workplace meetings, customer-service interactions and conversations at drive-through ordering points. For drive-throughs, the company uses recording hardware at the ordering point.

How the audio is captured

Vestergaard described several technical choices meant to help researchers train models:

• Each person is recorded on a separate audio channel, sometimes called a stem, so each voice can be analyzed independently.
• The channels are synchronized, allowing researchers to reconstruct the full conversation, including interruptions, overlapping speech and pauses.
• Audio is recorded at 48 kilohertz, meaning the system samples the sound 48,000 times per second, to preserve detailed sound information.

The common requirement across all settings, he says, is authenticity: people talking for a genuine purpose, such as solving a problem, sharing information, making a decision or completing a task.

Targeting a model's specific weakness

Vestergaard says the company starts by pinpointing exactly where a model struggles. Knowing that a model performs poorly at flight bookings, for example, is not enough. The underlying cause might be a misunderstood request, incorrect information retrieved, a failure to handle a correction, or trouble completing an action in another software system.

Once the failure is identified, the company looks for real situations in which people successfully perform the same task and collects those interactions. The dataset is built around the specific skill a model needs, he says, rather than simply gathering more speech on the same general topic.

Tone, pauses and interruptions carry meaning

Vestergaard argues that vocal features convey information beyond words. Someone saying "I love it" might be enthusiastic or sarcastic depending on tone. A pause might signal uncertainty or reflection, and an interruption might correct a misunderstanding or redirect the conversation. In his view, many speech datasets do not capture these signals in enough detail.

He expects this understanding to become essential as voice interfaces, systems that let people interact with computers through speech, become more common. He points to situations where mistakes carry consequences, such as emergency services and customer assistance.

What comes next for voice AI

The next challenge, according to Vestergaard, is teaching AI systems to speak, reason and act at the same time. He gives the example of a banking assistant that must keep up a conversation while checking records, evaluating information and deciding what to do next. To do that reliably, he says, models need examples of how people coordinate conversation, decision-making and action in real situations.

The company says its aim is to help developers build voice-AI systems that handle more complex tasks while communicating naturally. These are the company's own claims, made in an interview, and the source does not provide performance results for its datasets.