Brands Rate Their AI Support at 86%, but Only 51% of Customers Agree
Twilio says 86% of brands believe their customers are satisfied with AI support, while only 51% of consumers agree. In a blog post, the company argues the 35-point gap stems largely from how brands measure AI performance.
- Twilio says 86% of brands think their customers are satisfied with AI support, versus 51% of consumers who agree.
- Twilio research found 71% of consumers abandon a conversation if the AI fails to recognize them, and 49% say AI never resolved their issue.
- 73% of brands think customers want more human-like AI, but only 54% of consumers agree.
- Consumers rank immediate resolution and speed highest, both at 40%.
- Twilio recommends tracking abandonment by stage, repeat contact within seven days and containment split by outcome.
Why the numbers diverge
According to Twilio, most companies do measure their AI, but they look at the wrong things. The post points to three sources of bias in typical dashboards.
- Survey reach: Only completed conversations answer surveys. A post-interaction CSAT survey reaches people who stayed to the end, not those who gave up early. Twilio research found 71% of consumers will abandon a conversation entirely if an AI fails to recognize who they are, and almost none of those abandonments show up in a satisfaction score.
- Containment: A conversation the AI handled without escalating counts as contained, but it also counts as contained when the customer gave up and stopped replying. The two outcomes look identical in the data.
- Interactions versus progress: Brands score a session, while customers judge whether the thing they needed actually got done, including earlier attempts on other channels.
Twilio's report Inside the Conversational AI Revolution found that 49% of consumers say AI never resolved their issue.
What customers say they want
The disagreement extends to AI personality. Twilio reports that 73% of brands think their customers want AI to show more human-like personalities, yet only 54% of consumers agree. Nearly half are neutral or do not care whether the assistant sounds human, according to the post.
When consumers were asked what matters most in customer support, the top answers were practical:
- Immediate issue resolution: 40%
- Speed of assistance: 40%
- Accuracy of information: 38%
- 24/7 availability: 38%
Twilio notes that all four attributes are measurable, but a satisfaction survey of completed conversations will reveal little about them.
Metrics Twilio recommends adding
Twilio does not suggest dropping CSAT scores. It recommends layering on other measurements:
- Abandonment by stage: where in the conversation people stop replying.
- Resolution rate on the errand: whether the customer's underlying issue closed, including across sessions and channels.
- Repeat contact within seven days: described as the cleanest proxy for a first interaction that failed to fix the problem, requiring no survey.
- Containment split by outcome: separating conversations the AI resolved from those the customer exited.
- Escalation quality: how often a transferred customer has to re-explain their situation. According to the post, 76% say the human agent they reach has little or no context about them and their issue.
What it means for brands
Twilio's message is that the 86% figure is not a lie and that teams are not gaming metrics. It calls the number the honest output of a measurement system that only sees customers who stuck around.
The company suggests a simple first step: pull abandonment by stage for one month and compare it with the satisfaction sample. In Twilio's view, the distance between those two populations is probably the distance between 86 and 51. The post also promotes Twilio's tools for conversation management, including Conversation Memory, Conversation Orchestrator and Conversation Intelligence, as a way to narrow the gap between perceived and actual AI performance.
