Hallucination

Confident, fluent, wrong — and structurally unavoidable.

A language model produces plausible continuations. It has no separate faculty that checks whether what it is saying is true, which is why a fabricated case citation reads exactly like a real one: both are plausible continuations, and plausibility is the only thing being optimised.

The rate can be pushed down a long way — by retrieval, by citation requirements, by asking the model to say when it does not know, by checking outputs against a source. It does not reach zero, and a system that appears to have reached zero in testing has usually been tested on the cases it is good at.

The word itself is unhelpfully soft. Confabulation is closer: the model is not perceiving something that is not there, it is filling a gap with something shaped like an answer.

Why it matters here

The public sector exposure is specific: decisions must be defensible, and a fabricated authority in a brief, a statement of reasons or a piece of advice is a serious problem regardless of how it got there. The mitigation is process, not technology — a human who is accountable and who actually checks, and a design that makes checking fast rather than nominal.

The question to ask

What is the checking step, who does it, and how long does it take? If it takes longer than doing the work, the system saves nothing.

Reviewed 2026-09-20 · All decoders