Hugging Face Launches Open TTS Leaderboard for Multilingual Speech and Voice Cloning
Hugging Face has introduced the Open TTS Leaderboard, an open evaluation platform for comparing multilingual text-to-speech and voice-cloning models at scale. According to the blog post, it is meant to complement arena-style human voting, which struggles to keep up with the pace of new releases.
- Hugging Face introduced the Open TTS Leaderboard for multilingual text-to-speech and voice-cloning models.
- It aims to complement arena-style human voting, which struggles with fast model releases and hosting open models.
- Users can toggle languages to rank models on multilingual performance.
- Non-English and non-Chinese languages are scored on CV3 Eval (zero shot); CER is reported for Chinese, Japanese and Korean.
- The stated goals are reproducible comparisons and wider coverage of open models.
Why arena voting falls short
Traditional TTS arenas usually let users listen to outputs from two models and pick the one they prefer. Those votes are then turned into an Elo-style score, often with approaches such as the Bradley-Terry model. This gives users a simple way to see how systems perform based on human preference.
Hugging Face says human feedback remains an important part of judging speech quality. It points out several problems with relying only on voting, though. New TTS models are released at a rapid pace, which is hard for arenas to follow. Open-source models also face practical barriers, because they generally need to be hosted and served by the evaluation platform, while API-based commercial models can be added more easily. According to the post, this can leave open-weight systems less visible on public leaderboards.
Consistency is another concern. People's preferences and listening criteria can change over time, so votes collected months apart may not reflect exactly the same standards.
What the Open TTS Leaderboard does
The leaderboard is positioned as a scalable and transparent space for evaluating multilingual TTS and voice-cloning systems. Hugging Face lists three goals: make comparisons easier to reproduce, expand coverage of open models, and provide useful evaluation signals alongside human preference.
The post also stresses that language matters. In its words, "English performance doesn't necessarily translate to other languages." Users can toggle multiple languages to rank models on multilingual performance rather than on English alone.
Datasets and metrics
The post explains some details of how scores are produced. Seed TTS Eval only has audio for English and Chinese, so for the other languages the leaderboard uses the score on CV3 Eval, in a zero-shot setting.
Chinese, Japanese and Korean are character-based languages, so the leaderboard reports character error rate (CER) for them. The "Average WER" across languages is a macro-average across languages, meaning each language counts equally in the overall figure.
What it means for developers and users
For researchers and developers, the main promise is a way to compare speech models that does not depend only on who hosts them or when votes were cast. If the approach works as described, open-weight models could get more visibility next to commercial API-based systems. Hugging Face says that by combining scalable testing with community feedback, the project can help researchers, developers and users better understand how speech models perform across different languages, voices and use cases.
The source does not publish rankings or specific model results in the text available, so how models compare on the board is not covered here.
