Language models are trained to produce plausible continuations of text, not to look up facts. When they lack solid information, they can still generate a confident, well-formed answer that happens to be wrong, a behaviour often called hallucination.
The fluency is the trap: the same mechanism that makes answers readable makes mistakes read like truth. Guarding against it means checking claims against reliable sources, grounding answers in retrieved documents, and treating unverified output as a draft rather than a fact.