Refusal rate

The reason this metric is unpopular is that it looks like failure to everyone who has not thought about it. A refusal is a conversation the automation did not finish, and the natural instinct of anyone reporting on a deployment is to make that number small.

The instinct is exactly wrong. A refusal is the system correctly identifying the boundary of what it can support, and every refusal that reaches a person with the history attached is a conversation saved. The alternative is a confident wrong answer, which is not a smaller version of the same problem — it is a commitment made in your name to a customer who will hold you to it.

Watching it over time is where it earns its place. A refusal rate rising in one topic is a gap in the knowledge base with an address: those questions, that material, missing. It is the most actionable signal a deployment produces and the one most often switched off for looking bad.

Treating a refusal as a defect

Deployments get tuned until refusals disappear, and the tuning is reported as an improvement. What actually happened is that the boundary moved outward past the knowledge base, and every question that used to reach a person now receives an answer nobody can trace. The failure is now silent, which is worse than the failure it replaced.

Where this sits, and what it takes down with it.

Quantities derived from the counts. Each is only as sound as the definitions beneath it.

Cannot be computed without

Conversation

One exchange between a person and the system, from their first message to the point it is over — and where that point is decides every rate computed below it.

Somebody will quote this number at you.

Knowing the definition is not the same as being able to check the figure. These are the procedures that do the second thing.

Accepting an automated agent

“The agent is trained on your data and ready to go live.” · 45 minutes, 6 questions.

Answer rate

The share of conversations the system finished on its own — and on its own is doing more work in that sentence than it looks.

---

Refusal path

The route the system takes when it has nothing — designed, tested, and the single most valuable thing in a deployment.

---

Hallucination

A fluent, confident claim with nothing behind it — and the reason it is dangerous is that it is indistinguishable from a correct answer at the moment it is read.

---

Escalation rate

How often a person takes over — and whether they arrive holding the conversation or starting one.

All seventy-eight termsWhat this is regulated by in the UAE

Is this one right in your own numbers?

The definitions are the easy part. Whether the figure on your dashboard was computed this way is a different question, and usually the more expensive one.