ElevenLabs previews AI interaction tools at New York Summit

Domain-specific models and Interaction Engine

ElevenLabs said it is expanding beyond voice models and developing tools for the wider interaction layer between people and technology. The company previewed updates intended to make conversations with AI more natural and better suited to specific situations.

Its domain-specific Speech to Text models are designed to learn the terminology and norms used in particular industries. ElevenLabs highlighted Scribe v3 Medical, a transcription model for healthcare that can recognize thousands of complex medical terms and distinguish between drug names such as cycloserine and cyclosporine. The company said a domain-optimized model for financial services is also in development.

In the coming months, businesses will be able to fine-tune certain open-source language models and ElevenLabs voice models for their own use. Companies will be able to bring data from live deployments, choose metrics such as customer satisfaction score or reliability, and work with ElevenLabs researchers to refine the models.

ElevenLabs also previewed Interaction Engine. The system manages turn-taking and interruptions, and selects the right large language model for different parts of a conversation. According to the company, a less expensive model could handle triage while a frontier model takes on more complex tasks.

ElevenAgents adds memory and automated routines

ElevenAgents is ElevenLabs’ platform for configuring, deploying and improving voice and chat agents across sales, support, recruitment and operations. The company said customers and partners including Yahoo, Westgate and KPN are already using the platform.

A forthcoming Memory feature will let an agent build a record of a returning authenticated customer over time. It is intended to retain conversational details that may not have a field in a CRM, such as language preferences, scheduling constraints or an upcoming move. After each interaction, the agent distills the conversation into a structured record and updates it when the customer returns.

ElevenLabs also introduced Routines, an automation layer for work before, during and after conversations. Routines can connect with existing team tools and systems, and can be configured to pre-screen a mortgage applicant, remind patients about visits, offer canceled appointments to people on a waitlist, email a report, write to a CRM or start a call.

ElevenAI opens free early access

ElevenAI is a new app designed as an AI teammate for enterprise employees. It listens to spoken conversations, including in-person meetings, and turns them into notes and next steps. The app provides dictation and meeting notes using Scribe v3, which ElevenLabs describes as its newest and most accurate Speech to Text model.

With consent, ElevenAI can take notes during meetings and provide a summary and action items to each attendee. It can also offer live assistance during calls, sharing context and supporting collaboration between people and agents. ElevenLabs said notice and consent mechanisms are built in.

Free early access to ElevenAI is opening today. The company also said its partner network has grown to more than 550 partners in 44 countries, and announced the ElevenLabs Partner Program to help enterprises deploy ElevenAgents with consultancies, systems integrators, software vendors and agencies.

ElevenLabs CEO and co-founder Mati Staniszewski said conversational agents would become “critical infrastructure for every enterprise.” The announcements show the company targeting both sides of workplace interaction, with ElevenAgents focused on customers and ElevenAI supporting employees.