Adversarial test set

A demonstration shows the questions somebody chose. This is the opposite exercise: questions whose answers are absent from the corpus, questions whose answers are almost present, questions phrased like covered ones but asking something else, questions about a competitor's product, questions with a false premise embedded in them, questions about prices in a currency you do not quote.

It is written once against the material, run on every change, and kept as a file rather than a memory. The results are a table anybody can read: asked, answered, refused, escalated, invented. The last column is the only one that matters and it should be zero on anything with a number in it.

It is also the right thing to require at acceptance, which is why this practice ships it as a deliverable rather than an internal artefact. A supplier who will not hand over the set that broke their own system has withheld the only evidence a buyer could have used.

Accepting a demo instead

The demo is the supplier driving. Ask instead for the adversarial set and its results, then add ten questions of your own that nobody has seen — the ones your staff dread on the phone. A system that handles the supplier's set and invents on yours has been tested against the wrong thing, and it is better to find that out before the invoice.

Somebody will quote this number at you.

Knowing the definition is not the same as being able to check the figure. These are the procedures that do the second thing.

Accepting an automated agent

“The agent is trained on your data and ready to go live.” · 45 minutes, 6 questions.

Refusal path

The route the system takes when it has nothing — designed, tested, and the single most valuable thing in a deployment.

---

Hallucination

A fluent, confident claim with nothing behind it — and the reason it is dangerous is that it is indistinguishable from a correct answer at the moment it is read.

---

Grounding

Every sentence the system produces is traceable to a passage it retrieved — which means an answer can be checked without asking the model to explain itself.

---

Handover

The point at which a vendor's work becomes the company's property — defined by what is transferred, not by the project being finished.

All seventy-eight termsWhat this is regulated by in the UAE

Is this one right in your own numbers?

The definitions are the easy part. Whether the figure on your dashboard was computed this way is a different question, and usually the more expensive one.