Closed by the chatbot. Still open for the customer.

The conversation ended without a request for a human. Your dashboard counts it as contained.

Perhaps the customer got what they needed. Perhaps they gave up and called their salesperson.

Those possibilities produce very different business outcomes. If the reporting cannot distinguish them, a rising containment rate is difficult to interpret.

Imagine a customer trying to change a delivery address after the parcel has been handed to the carrier. The assistant directs them to the address field in their account. They save the new address, then ask whether it applies to the parcel already travelling. The assistant repeats the account instructions. The customer leaves the chat and calls their salesperson with the tracking reference.

The chat is closed. The work has moved.

Follow a few closed cases beyond the chat window

Start with a small, authorised sample of closed conversations. Choose several types of request rather than only the shortest or most positive exchanges. Respect the access and retention rules that apply to customer records.

For each case, ask what completion would actually mean. A document request may end when the correct document is delivered. For this address change, saving the customer’s new account address does not establish that the carrier will reroute the parcel. The case needs the carrier’s acceptance or refusal of the rerouting request, and a reply to the customer explaining where the parcel is now expected to go. A refund question may require a recorded decision rather than another description of the policy.

Then examine the available evidence. Did the underlying record change? Did the customer contact support again? Was the issue reopened through another channel? If those systems cannot be linked lawfully and reliably, record the outcome as unknown. Do not turn a gap in tracking into a success.

This exercise is not a statistically representative customer study. It can reveal specific failure paths worth investigating. Keep that distinction when presenting the result.

Find the moment the customer stopped making progress

Read the conversation as a sequence of attempts.

The first answer might be perfectly appropriate for a different state of the order. The second might repeat it after the customer explains that the parcel has already left. A third request for an order number may add no information because the customer supplied it at the start.

Identify the first point where the assistant had enough evidence to change its approach. That is more useful than labelling the whole conversation “bad”. It gives the implementation team a concrete behaviour to test.

In the delivery example, the important distinction is the dispatch state. Once the order has shipped, the ordinary edit instructions no longer resolve the request. The assistant needs the agreed post-dispatch route, which may involve a carrier, an internal team or an honest refusal to promise a change.

Do not assume every unresolved conversation needs another AI model. Sometimes the process itself has no defined route for the customer's situation.

Let the customer reach a person without solving your routing puzzle

A visible route to help is useful even when the assistant believes it has answered.

The customer may have a constraint that the system has not understood. They should not need to guess the phrase that triggers escalation or repeat a complaint several times before the route appears.

The right arrangement depends on the service. It might be a callback request, an existing support queue or a direct transfer during staffed hours. Be precise about availability. A button that implies immediate help but deposits a message in an unattended queue creates a different disappointment.

Where the service has response targets, show the target the team can support. Do not invent a reassuring wait time because the interface needs a sentence.

Carry the conversation across the boundary

A transfer that loses the customer's work is expensive in attention, even if the routing itself is fast.

The person receiving the case should have the request, relevant identifiers, what the assistant tried and what remains unresolved. The customer should know that the transfer happened and what comes next.

Be careful with summaries. They can help a colleague read quickly, but the original exchange must remain available where permitted. A summary that says “customer wants delivery information” may conceal the actual issue: the order is going to an address the customer can no longer access.

The person also needs to know whether the assistant performed any action. A drafted change, a submitted change and a confirmed change are not interchangeable. That status affects what the human should do next.

“Won't this send too much work to the team?”

It may reveal work that the chat dashboard was not counting.

That does not mean every customer request should immediately go to a person. Good self-service should help people finish routine tasks without waiting. The question is whether the automated path offers a real outcome for this case and whether the customer can leave it when it does not.

Measure the resulting human work rather than assuming either that escalation is free or that avoiding it creates savings. Look at repeat contacts, unresolved cases and the effort required after transfer. If the same issue repeatedly needs a person, examine whether the assistant lacks information, the process lacks a decision or the task should remain with trained staff.

The team may discover a useful automation opportunity in that evidence. It may also discover a confusing policy that needs a plain-language correction.

Test with a customer who cannot follow the standard answer

Use a safe simulation. Start with a routine request, then add a condition that makes the standard route unsuitable. In the delivery example, the shipment has already left.

Check whether the assistant notices the condition, avoids promising an action it cannot confirm and offers the agreed next step. Follow the case into the human queue. Verify that the recipient can continue without asking the customer to start again.

Then inspect how the result appears in reporting. A transferred case should not quietly become a resolved case merely because the bot stopped speaking.

The team has time now. The customer is still waiting looks at the wider operating decisions behind customer outcomes. This is one place to examine them closely: a conversation marked complete while the customer's request remains open.

Bring a few anonymised conversations to a Strategy Session. Include one that looked successful in the dashboard but needed help elsewhere.