Hallucination: a plausible answer can still be wrong

An AI hallucination is generated content that is false or unsupported by the supplied information. A model can produce a convincing explanation, reference or quotation without having reliable evidence for it.

Next-token prediction rewards patterns that fit the training distribution. Fluent language and correct facts often coincide, but the objective does not turn every generated statement into a checked observation. Training text can also contain misconceptions the model learns to repeat.

Confidence has several meanings

A high token probability is confidence about a continuation under the model. It is not the same measurement as the probability that a whole claim is true. A model's written confidence is another output, with its own calibration limits.

For example, a citation can have the right-looking author, year and title while referring to no real publication. The format is easy to imitate; verifying that source exists is a separate operation.

Evidence changes what can be checked

RAG or tool calls can supply external observations. That creates a basis for checking claims, but the model can still misread a passage, combine incompatible sources or attach a real citation to an unsupported statement.

Useful evaluation asks concrete questions: does the source exist, does the quoted text match, does the passage support the claim, and does the claim hold in the requested setting?

There is no single surface feature that reliably separates truth from invention. A polished answer may be wrong; an uncertain answer may be correct. The interesting engineering problem is connecting important assertions to observations that can actually be verified.