Hallucination, Confabulation, and Why the Word Matters
The field calls all of this “hallucination,” and the word has been stretched until it covers almost every way a model can be wrong. Sebastian Farquhar and colleagues, in a 2024 Nature paper, make the useful point that there’s no reason to expect a single mechanism behind every kind of error, so lumping them under one word hides more than it reveals. The standard surveys (Huang and colleagues, 2025) split the term along two axes. One is faithfulness: whether the output stays true to the source you gave the model. The other is factuality: whether the output is true about the world. A summary that contradicts the document it’s summarizing is a faithfulness failure. A confident claim about a person who doesn’t exist is a factuality failure. They look similar on the surface and come from different places.

This post is about one specific kind, and it has a better name: confabulation. The word comes from clinical psychology, where it describes a person who fills a gap in memory with a fabricated account, told sincerely and with full confidence, and with no intent to deceive. That last part matters. The person isn’t lying, because lying requires knowing the truth and choosing to hide it. They simply have a hole where a memory should be, and the mind produces something plausible to fill it. “Hallucination” implies perceiving something that isn’t there, which is the wrong picture for a language model. “Confabulation” implies filling a gap with a confident guess, which is exactly right. Farquhar’s group uses the term in a precise way, for the subset of errors that are arbitrary: answers that change if you run the model again with a different random seed, because there was never a stable fact behind them in the first place. I’ll use it the same way.

The intuition is the student who didn’t study but refuses to leave the answer blank. Asked a question they can’t recall, they write something that sounds like the textbook, in the right tone, with the right shape, and sometimes they’re even close. The model does this constantly, and it does it well, because producing text that sounds right is the one thing it was built to do.

Why It Happens
A language model is trained to predict the next token, the next chunk of text, given everything before it. That’s the whole objective. It does not have one mode for recalling a stored fact and another for making something up. It has a single mode: produce the most probable continuation of this text. When the fact you’re asking about appears often and consistently in the training data, the most probable continuation happens to be the true one, and the model looks like it’s remembering. When the fact is rare, or absent, or you’ve asked about something that was only ever stated once, the most probable continuation is still a fluent, well-formed answer. It just isn’t tied to anything true. The model has no way to feel the difference, because from the inside both cases are the same operation.