Refusal path

Every one of these systems will meet questions its material does not cover, on the first day and every day after. What happens then is a design decision, and where nobody has made it the default takes over: the model produces something fluent, because that is what it does when given a question and no answer.

A working refusal has three parts and all three are usually missing. The system has to be able to know it has nothing, which means retrieval must be permitted to return empty rather than always returning its best three passages. It has to say so in a way that keeps the conversation alive rather than ending it. And it has to hand over with the history attached, so the person arriving is continuing something rather than starting fresh.

This practice states it as the invariant on the service that sells these: an agent that will not say it does not know is not an asset. It is a liability that has not yet been triggered.

Building it and never testing it

The refusal path is the least-exercised branch in the system and the one whose failure is most expensive, which is a poor combination. It needs its own test set — questions with no answer in the corpus, questions whose answer is nearly present, questions phrased to sound like ones that are covered — run on every material change. A path nobody has fired is not a path.

Somebody will quote this number at you.

Knowing the definition is not the same as being able to check the figure. These are the procedures that do the second thing.

Accepting an automated agent

“The agent is trained on your data and ready to go live.” · 45 minutes, 6 questions.

Refusal rate

How often the system says it does not know — the one metric on a deployment where zero is the alarming reading.

---

Hallucination

A fluent, confident claim with nothing behind it — and the reason it is dangerous is that it is indistinguishable from a correct answer at the moment it is read.

---

Escalation rate

How often a person takes over — and whether they arrive holding the conversation or starting one.

---

Adversarial test set

The questions written specifically to make the system invent an answer — and the only acceptance evidence that means anything.

All seventy-eight termsWhat this is regulated by in the UAE

Is this one right in your own numbers?

The definitions are the easy part. Whether the figure on your dashboard was computed this way is a different question, and usually the more expensive one.