UW study finds AI models score some accents lower in speech evaluations
A University of Washington-led study found that leading AI models rated Chinese and Nigerian English accents lower than American and British accents when evaluating interviews, presentations and spoken English tests. The models often favored those accents even when the spoken content was identical and accurately transcribed.
- UW researchers tested Gemini 2.5 Pro, Gemini 2.5 Flash, GPT-Audio 1.5 and Qwen 3 Omni.
- Chinese and Nigerian accents generally received the lowest ratings.
- American and British accents received the highest ratings.
- The study used identical scripts and validation from 52 human speakers.
- Researchers plan to examine more accents and medical audio applications.
Four models evaluated five English accents
The researchers tested three proprietary models, Google’s Gemini 2.5 Pro and Gemini 2.5 Flash, and OpenAI’s GPT-Audio 1.5, alongside the open-source Qwen 3 Omni. The models rated speech on a scale of one to seven for both delivery and content, and provided short explanations for their scores.
AI voices from ElevenLabs read the same scripts in American, British, Chinese, Indian and Nigerian accents. The scripts covered job interviews, class and conference presentations, and spoken English proficiency tests. The team also validated its findings with 52 human speakers from the five countries.
American and British speakers received higher ratings
The models generally ranked Chinese and Nigerian accents lowest, Indian accents in the middle, and American and British accents highest. They rated both delivery and content more favorably for American and British speakers, even when the words and underlying responses were the same.
The human interview evaluation also favored American and British accents, with a larger gap between human speakers than between the AI voices. Researchers found that accents with greater acoustic distance from American accents were often ranked lower.
The models’ explanations reflected the same pattern. They repeatedly described Chinese and Nigerian speakers as having “mumbled” or been “difficult to understand,” although the systems were able to transcribe the accented speech accurately.
“They understood the speakers, but they just had this implicit notion of hierarchy among them based on accent,” co-lead author Orevaoghene Ahia, a UW doctoral student, said in the UW News report.
The findings raise concerns for hiring and education
The research was presented Oct. 25 at the Conference on Empirical Methods in Natural Language Processing in Budapest, Hungary. Ahia said audio AI models are increasingly being trusted as third-party evaluators in hiring, oral education exams and English tests used in immigration, where fairness matters.
The researchers said accent bias could enter models through training data that describes accented speakers as difficult to understand, underrepresents some populations, or reinforces conventional ideas about how a professional speaker should sound. The study also points out that English is the official language in Nigeria and the teaching language in Indian schools, while those accents still received lower ratings than British and American accents.
Senior author Yulia Tsvetkov said future work should create more representative training and evaluation data sets. The team plans to study more accents, including regional variations within countries, and audio analysis in medical contexts such as automated emergency response.
