Hallucination

It is not a bug that will be fixed, and treating it as one leads to the wrong architecture. Generating plausible continuations is what these systems do; producing a plausible continuation when the material is absent is the same behaviour, working as designed. The engineering question is therefore not how to stop it but where to put the boundary that catches it.

The commercially expensive cases are narrow and predictable. Prices, delivery times, eligibility, refund terms and anything with a number in it — invented confidently, at scale, in writing, to customers who will act on it. Nobody is harmed by a chatbot being vague about company values.

Which is why the answer is structural rather than tonal. Retrieval that can return nothing, a refusal path that fires when it does, a deterministic layer holding a veto over anything with a price in it, and an adversarial test set that keeps trying to make it invent. Prompting a model to be careful is not one of the controls; it is a request.

Trying to prompt it away

Instructions like “only answer from the provided context” and “say you don’t know if unsure” reduce the frequency and do not change the class of failure, because the model has no reliable access to whether it knows. Anything the business would be held to needs a check outside the model — a rule, a lookup, a required source — and the check has to be able to block the reply.

Somebody will quote this number at you.

Knowing the definition is not the same as being able to check the figure. These are the procedures that do the second thing.

Accepting an automated agent

“The agent is trained on your data and ready to go live.” · 45 minutes, 6 questions.

Grounding

Every sentence the system produces is traceable to a passage it retrieved — which means an answer can be checked without asking the model to explain itself.

---

Refusal path

The route the system takes when it has nothing — designed, tested, and the single most valuable thing in a deployment.

---

Deterministic layer

The code that holds a veto over what the model produces — rules that always run and always give the same answer to the same input.

---

Adversarial test set

The questions written specifically to make the system invent an answer — and the only acceptance evidence that means anything.

All seventy-eight termsWhat this is regulated by in the UAE

Is this one right in your own numbers?

The definitions are the easy part. Whether the figure on your dashboard was computed this way is a different question, and usually the more expensive one.