Krisp Recorded 265 Real Conversations to Test Background Voice Handling

Three months of recording in real places

The benchmark consists of recordings made in offices, call centers and cars, on the devices people actually use. Each one was then transcribed and labeled segment by segment by hand. Krisp says the work took three months and involved 24 audio devices in the office scenario alone, 18 headset models across four call-center floors, and 18 phone models, plus whatever Bluetooth microphone was in each car.

The Krisp Engineering Team describes a messy process. Four working call centers had to agree to host the recording team, and sessions had to fit around their shifts and customers. Eighteen headset models meant eighteen different connectors, and on more than one occasion the team arrived without the right adapters. Many takes were discarded because the second speaker started too early, stopped too soon, or was too quiet or too loud. Others were lost to a muted microphone or the wrong input.

For the car recordings, a colleague named Anna asked nearly everyone at Krisp to record scripts while driving, and attached the line "eyes on the road, the script can wait" to each request. By the team's count she sent it 41 times. According to Krisp, nobody crashed.

Why Krisp did not simulate the audio

The common way to build an evaluation set is to take clean speech, add a noise recording and an impulse response, and treat the result as a room. Krisp says this works for training but not for evaluation, because the thing most worth measuring is the degradation nobody thought to simulate.

The company argues the problem matters more as people increasingly talk to bots. When a secondary voice leaks into the channel, the speech-to-text model transcribes it alongside the intended speaker, and the transcript then reaches the LLM, whose behaviour can shift. Krisp gives the example of a customer saying "I'd like to cancel my subscription" while a colleague nearby says "Can you grab lunch?", with the bot hearing both.

Krisp says modern speech-to-text has made real progress on noise such as traffic, air conditioning and keyboard clatter. Background speech is harder, it argues, because a background voice shares the acoustic structure of the target speaker, so models transcribe every voice rather than dismissing the secondary one. The company also points to a gap in data for degraded audio such as Bluetooth car microphones and low-bandwidth telephony codecs, where an isolation model has little room to clean the signal without damaging it.

Three scenarios in the dataset

The recordings are split into three categories:

• Work: open-plan offices, home and meeting rooms, with 29 speakers, 96 recordings, 24 devices, 4 room types and 17 scripts. Devices range from professional Jabra headsets to consumer earbuds.
• Call center: collected at four real locations, with 7 speakers, 106 recordings, 18 headsets and 28 scripts. Scripts covered banking, insurance, healthcare, telecom and more. Background voices here are constant and structured, each agent carrying their own customer conversation.
• Phone calls: 26 speakers, 63 recordings, 18 phone models, 4 location types and 20 scripts. Most were made in cars over a Bluetooth car microphone, on speakerphone or through earbuds, with others from noisy buildings, shops, streets and metro stations. Here the challenge is the degraded audio itself rather than a deliberate second speaker.

In the Work and Call Center scenarios, noise such as office chatter, keyboards, music and babble appears naturally alongside the secondary speakers. Krisp says this variability is what synthetic mixing cannot replicate.

Labels, results and what it means

Each recording comes with a hand-written transcript and a JSON metadata file containing the transcript text, speaker gender, recording device and recording environment. The transcripts capture speech as it occurs.

Krisp reports that where a second voice is present, its Voice Isolation product cuts word error rate by roughly three quarters. On already-clean phone audio, it says, the processing costs a little more than it returns. These are the company's own claims about its own product.

For teams building voice agents or evaluating speech-to-text, the main point is that background speech is a distinct failure case that existing public benchmarks for noise, reverberation and accents do not cover. A dataset recorded in real rooms on real devices is intended to show how systems behave when the room is not quiet.