ElevenLabs launched ElevenAI, a new workplace app, along with custom model fine-tuning and a formal partner program at its New York Summit on Wednesday. The voice AI company also previewed an Interaction Engine for managing live conversations and new Memory and Routines features for its ElevenAgents platform.
“Within a few years, conversational agents will become critical infrastructure for every enterprise, as essential as cloud or payments”— Mati Staniszewski, CEO and co-founder of ElevenLabs
ElevenAI listens to spoken conversations, including in-person meetings, and produces notes, summaries and action items. Free early access opened with the announcement. The app runs on Scribe v3, ElevenLabs' transcription model, for both dictation and meeting notes.
The fine-tuning service starts in the coming months. ElevenLabs will work with businesses to fine-tune selected open-source language models and its own voice models using data from live deployments, with customers choosing the metric to optimize, such as customer satisfaction or reliability. The company already offers Scribe v3 Medical for healthcare transcription and said a model for financial services is in the works.
Key facts
- 01ElevenLabs opened free early access to ElevenAI, an app that turns conversations into notes and action items.
- 02In the coming months it will fine-tune select open-source language models and its voice models on customer data.
- 03ElevenAgents gains Memory for returning customers and Routines for multi-step automation.
- 04The new Partner Program spans more than 550 partners in 44 countries, including Google Cloud, IBM and Deloitte.
- 05ElevenLabs is adding 15 MW of Nvidia Vera Rubin compute capacity.
The Interaction Engine handles turn-taking and interruptions and picks which language model handles each part of an exchange, so a cheaper model can triage a request while a frontier model takes the complex steps. ElevenLabs did not give an availability date for it.
ElevenAgents is getting two additions. Memory keeps a record of returning customers and their preferences, and Routines is a new automation layer for multi-step work such as mortgage pre-screening, patient reminders and waitlist offers.
“One of the hallmarks of a great conversation with a person is that they remember what you told them and act on it. Agents should do the same”— Mati Staniszewski, CEO and co-founder of ElevenLabs
On distribution, ElevenLabs formalized a Partner Program covering a network of more than 550 partners in 44 countries, including Google Cloud, IBM, Deloitte, DXC and Genesys. It also announced an alliance with McKinsey and QuantumBlack, AI by McKinsey, and launched a Marketplace where businesses will be able to buy agents starting next year.
The company expanded its work with Nvidia as well. Nemotron 3.5 Super VL is now available in ElevenAgents, and ElevenLabs is adding 15 MW of new Vera Rubin compute capacity. It also plans an on-premises voice agent for Poland's national e-Health platform.
CEO and co-founder Mati Staniszewski framed the fine-tuning push as a way for companies to keep what makes them distinct. "We will not solve human-AI interaction without reflecting what makes people and businesses distinctive," he said.
Working on something we should cover, or seeing a story we missed? Send leads, documents, or feedback to hello@aichatdaily.com. For sensitive tips, see our secure tips page for Signal and PGP options.
Spotted an error? Email hello@aichatdaily.com with the URL and the issue, or read our full corrections policy.
