The measured evidence on AI for customer experience in call centres shows a split. In the measured studies, AI sped handling up (faster issue identification, shorter chats) and lifted customer ratings. In the Alibaba field experiment, though, it left the objective measure unchanged: whether the customer comes back with the same problem. That split shapes how an efficiency gain should be proven. If you only track speed and satisfaction scores, a deployment can look like a success while repeat contacts, and the cost they carry, stay where they were.
This article sets out which metrics move and which do not, what to baseline before go-live, how to judge the AI apart from the agents using it, and where the cost saving runs out.
What AI in call centres moves, measured
Two large studies give causal evidence. Brynjolfsson, Li and Raymond (QJE, 2025) followed the staggered rollout of a generative AI assistant across 5,179 support agents and found issues resolved per hour rose 14% on average, with a 34% improvement for novice and low-skilled agents and minimal impact on experienced, highly skilled ones. A randomised field experiment in Alibaba’s e-commerce after-sales chat (arXiv, 2026) gave human agents a gen AI assistant that suggested diagnoses and solutions, which they could adopt, edit or ignore.
| Metric | Direction in the measured studies | Source |
|---|---|---|
| Issue identification time | Down 8.2% with access; down 32.3% at full usage | Alibaba field experiment (2026) |
| Chat duration (handle time) | Down 1.1% with access; down 4.2% at full usage | Alibaba field experiment (2026) |
| Customer rating | Up 0.042 points on a five-point scale (1.2%) | Alibaba field experiment (2026) |
| Dissatisfaction rate | Down 0.012 (3.4% relative) | Alibaba field experiment (2026) |
| Three-day retrial rate (same issue) | Near zero, not statistically significant | Alibaba field experiment (2026) |
| Issues resolved per hour | Up 14% average, 34% for novices | Brynjolfsson et al. (2025) |
| Customer sentiment, agent retention | Improved | Brynjolfsson et al. (2025) |
The two studies measured different service processes. The Alibaba authors note that Brynjolfsson et al. measured B2B technical support, where interactions are long and productivity is counted in resolutions per hour, whereas Alibaba’s setting is short, high-volume, concurrent chats under standardised procedures. The same tool can raise throughput in one operation and mainly change the feel of the interaction in another.
Faster handling did not reduce repeat contacts
The Alibaba experiment shows that shorter chats did not by themselves reduce repeat contacts for the same issue. Its pre-experiment baseline was 35.53 seconds to identify the issue, 465.62 seconds per chat and 38% of customers returning with the same issue within three days. The assistant shortened the first two figures and did not move the third.
The mechanism is mundane. Alibaba’s standard operating procedures define what agents are authorised to do, those rules did not change, and the assistant did not widen the set of available solutions. An AI that helps an agent diagnose faster and phrase replies better cannot resolve an issue the agent was never allowed to resolve.
Satisfaction moved where resolution did not. Ratings rose and dissatisfaction fell while retrials stayed flat, which is why the authors recommend judging gen AI on both perception-based and resolution-based outcomes.
A 2025 telesales paper cites a 35% handle-time reduction alongside a 30% satisfaction gain, attributing both to a referenced source rather than its own trials. We would not plan a voice deployment on that number.
Uniform rollout can make your best agents worse
The average effect hides a split by skill. In the Alibaba experiment, agents in the lowest pre-treatment quintile saw dissatisfaction fall by 0.215 and ratings rise by 0.812 points, while top-quintile agents saw dissatisfaction rise by 0.070, ratings fall by 0.283 points and three-day retrials rise by 0.009. The authors offer workflow disruption as a possible explanation: for top performers, AI use increased shift-away time and response time.
Top performers also used the tool less, with usage falling from 39.3% for the lowest quintile to 18.3% for the highest. The practical conclusion the authors draw is to preserve discretion for top performers and watch whether AI use disrupts service continuity, rather than rolling it out uniformly. This matches the Brynjolfsson finding that the gains concentrate among novices. If you pilot with your strongest team, you may measure the smallest gain, or a loss.
KPIs to baseline before the AI goes live
Proving an efficiency gain needs a pre-AI baseline that covers resolution as well as speed. CloudTalk, a voice AI vendor, advises pulling at least three months of historical data before deployment, covering AHT, CSAT, FCR, cost per contact, repeat rate and attrition. MetricNet proposes five metrics every contact centre should track whatever its AI maturity: contact deflection rate, agentless resolution rate, cost per contact, first contact resolution and customer satisfaction, as a basis for comparing the AI-enabled operation with a non-AI baseline.
A baseline checklist we would use:
- Average handle time, split into talk, hold and after-contact work, since Amazon Connect, for example, defines AHT to include all three.
- First contact resolution, measured from the customer side as well as the system side (see the caveat below).
- Repeat contact rate within a fixed window, since a three-day same-issue retrial is the objective quality measure the Alibaba study used.
- CSAT and dissatisfaction rate, kept separate, as the Alibaba study did.
- Cost per contact, fully loaded, meaning total centre cost divided by all contacts including those resolved in agentless channels, per MetricNet’s definition.
- Deflection and agentless resolution, recorded as two different numbers.
- Agent performance distribution, so effects can be read by skill band rather than as one average.
The FCR caveat matters. The COPC 2022 benchmarking survey found organisations overestimate FCR compared with customer-reported data, in both assisted and self-service channels, often because they measure it as the share of customers who did not call back within 24 hours. A baseline built on an inflated FCR makes any later change hard to read.
Judging the AI separately from the agents
AI performance and agent performance need separate scorecards, because a good assistant can sit in a poorly run queue and a weak one can be masked by strong agents. Contact-centre platforms such as Amazon Connect document AI-side metrics.
For AI agents that handle contacts themselves, Amazon Connect’s documentation defines an AI handoff rate (the share of AI-handled contacts escalated to humans), invocation success excluding technical failures, invocation latency in milliseconds, and average conversation turns to reach an outcome. It also defines LLM-judged quality scores: goal success rate, completeness, a faithfulness score that checks responses against the conversation for hallucination, and a context fidelity score for whether the AI passed accurate context to the downstream agent.
For agent-assist tools, one signal is whether agents actually use what the AI suggests. Google Cloud’s documentation shows how to compute an effective click-through rate for smart replies and a summarisation edit distance between suggested and agent-revised summaries. The Alibaba data shows why usage belongs on the scorecard: at chat level the assistant was used in 21.5% of chats, with unmodified adoption in 17.1%.
Two further methods are worth borrowing. FECT, a benchmark accepted at a KDD 2025 workshop, evaluates the factuality of AI-generated claims about contact-centre transcripts. A telesales study scored an AI voice agent and human agents blind, with seven evaluators and a 22-criterion rubric. The AI matched humans on introduction and product communication but lagged on objection handling and sales drive.
What drives cost per contact down, and what caps the saving
Labour drives the cost, and a contact the AI handles without resolving it is a deferred cost rather than a saving. Gartner (2022) puts labour at up to 95% of contact-centre costs. Its 2019 poll of customer service leaders put live channels at an average of $8.01 per contact against about $0.10 for self-service.
The same 2019 Gartner research shows the limit: only 9% of customers reported resolving their issue completely through self-service, and those who cannot then move to live channels, raising operating costs. A contained contact that ends in a callback has not saved the gap between those two averages. It has moved the work into a repeat contact. CloudTalk calls a handled-but-unresolved call a deferred cost and argues for cost per resolved call as the stricter measure. The same source warns that containment is not resolution and should always be read alongside repeat contact rate. MetricNet sets the same rule for its agentless resolution rate: a contact counts only when the customer’s need is completed, not when it was merely contained or abandoned.
Partial automation also has value. Gartner (2022) notes value in partial containment, such as automating identification of the customer’s name, policy number and reason for calling. The costs on the other side are real: Gartner estimates integration at $1,000 to $1,500 per conversational AI agent, with some organisations citing up to $2,000. The COPC CX Standard 7.0 observes that human channels carry high operating costs and digital channels high capital costs, and only blending the two shows whether migration really lowers cost per unit. It also recommends that savings calculations be conservative, monetary, annual, split between one-time and ongoing, and backed by data.
Does AI in a call centre replace human agents?
The measured study shows AI raising agent productivity, while Klarna has resumed hiring human agents and CommBank reversed its contact-centre job cuts. Klarna said in February 2024 that its assistant handled two-thirds of customer service chats, did the work of 700 full-time agents, matched human satisfaction and cut repeat inquiries by 25%. Klarna’s CEO later said the chatbots, while cheaper, produced lower-quality output, and the company began recruiting human agents again. Commonwealth Bank of Australia cut 45 contact-centre jobs and then reversed the decision after its voice-bot increased human workload, admitting it had not adequately considered all relevant business factors, as Finextra reported.
Capacity gains without headcount cuts are the more defensible pattern. In a healthcare call-centre deployment, the AI agent managed over half of phone-scheduled appointments, at 850 weekly talk hours, equivalent to 28 full-time specialists. Patient refusal stayed below 3% and the centre handled twice the workload without extra staff. That was clinical outreach and scheduling, so the transfer to a general service queue is a reasonable inference, not a finding. Gartner (2025) predicts agentic AI will autonomously resolve 80% of common service issues by 2029. That is a forecast, not a result.
Which call-centre AI improvement gives the fastest payback?
Our judgement is that agent assist for newer and lower-performing agents, plus partial automation of the identification steps at the start of a contact, gives the most dependable early return. The reasoning comes from the evidence: the largest measured productivity gains went to novices, issue identification time fell for every skill band in the Alibaba experiment while chat duration fell significantly only for the first and third quintiles, and Gartner (2022) places value in partial containment.
Full end-to-end automation counts as a saving only where the AI resolves, not merely handles, the call. Otherwise the work moves into repeat calls. Our article on where chatbot productivity gains are real looks at that boundary from the self-service side.
Is cloud contact-centre software necessary for AI features?
What AI features actually need is access to interaction data and an integration point in the contact flow. Amazon Connect documents an AI handoff rate, and Google Cloud shows how to compute metrics from Agent Assist conversation data. Gartner (2022) prices conversational AI integration per agent and expects organisations with 2,500 or more agents to lead adoption. The deciding factor is whether you can instrument handoffs, resolution and repeat contacts on your stack, not where it runs.
How does AI handle multilingual customer support?
Large language model assistants can serve many languages from one deployment: Klarna’s bot handled refunds, returns and payments in more than 35 languages. Language reach also widened access in one outreach programme. In a multilingual cancer-screening outreach programme, Spanish-speaking patients connected at 69.6% against 53.0% for English speakers and opted in at more than twice the rate. Coverage is not quality, though, and Klarna’s later quality concerns are a reminder to evaluate each language with the same faithfulness and resolution metrics you use for the main one.
What is the biggest risk when deploying AI in a call centre?
The biggest risk is declaring an efficiency win on speed and containment while resolution stays flat, then cutting capacity on that basis. The CommBank reversal shows the failure, with call volumes rising after a bot meant to reduce them. The Alibaba results show how it happens without anyone misreporting: handle time and ratings improved while three-day retrials did not. MetricNet observes that centres relying on scorecards built for human operations can report AI activity but cannot show AI value.
The protection is the pairing described above: baseline repeat contacts before go-live, count savings per resolved contact, and score the AI on its own handoffs and faithfulness. The question you still have to settle is whether your rules let the AI, or the agent it assists, resolve more. If they do not, faster handling is the most you will get.
