Last reviewed:
What is an AI hallucination? Definition and pitfall to avoid
A hallucination is a factually false statement but formulated with confidence by a generative AI model. It stems from the statistical mechanism of generation: the model predicts the most likely next word, with no native capacity to verify. It is the main operational and legal risk of an enterprise AI deployment.
Hallucination is not a bug, it is a structural property of LLMs. Four mechanisms produce it. Excessive generalisation: the model extrapolates a plausible response from patterns seen in training, even when the precise topic was not covered. Interpolation between contradictory sources: the model merges diverging information into a smooth but inaccurate statement. Pressure to respond: without guardrail, the model prefers to generate a confident response rather than acknowledge ignorance. Temporal mismatch: models trained before a given date are unaware of subsequent developments (knowledge cutoff). Three factors significantly reduce hallucinations: use of inference-time retrieval (RAG or web search), explicit instruction to the model to flag uncertainties, and systematic human validation on outputs with stakes. None eliminates the phenomenon entirely. According to 2025-2026 benchmarks, the best models still hallucinate on 5 to 15% of complex factual queries.
Concrete example
Moffatt v. Air Canada case, February 2024. A customer queries Air Canada's AI chatbot on bereavement fare policy. The chatbot invents a retroactive refund procedure within 90 days, whereas the company's real policy requires booking at normal fare then refund request before travel. The customer follows the chatbot's instruction, pays the normal fare, and is refused the refund. He sues Air Canada before the Civil Resolution Tribunal of British Columbia, which condemns the company in February 2024 to honour the policy invented by its own chatbot. First major Anglo-Saxon legal precedent on the editorial responsibility of an AI chatbot.
A second case, in a professional context: Mata v. Avianca, June 2023, US federal court in New York. Two lawyers from Levidow, Levidow & Oberman filed a 10-page legal brief citing six court decisions — all fabricated by ChatGPT. When the lawyers asked the model to confirm the citations were real, it "confirmed" they were. Judge Kevin Castel fined them $5,000 each for filing in bad faith, ordered a public professional sanction, and notified their clients. The case became a reference precedent: US, UK and Canadian courts now explicitly require human verification of AI citations in filings.
Comparison
| Technique | Effectiveness | Implementation cost | Typical use case |
|---|---|---|---|
| RAG (Retrieval-Augmented Generation) | High on facts grounded in a clean document base | Medium — vector infrastructure + continuous source curation | Assistant answering on a company knowledge base (support, HR, legal) |
| Chain-of-verification (self-check) | Medium — reduces errors but token-expensive | Low — prompt modification only | High-stakes content generation (summaries, reports) with no external data |
| Grounding + mandatory citation | High — the model is constrained to source every claim | Low to medium — prompt instruction + output post-processing | Analytical reports, document summaries, legal answers |
| Human-in-the-loop (systematic review) | Maximum — no unvalidated output reaches production | High — human bottleneck, doesn't scale beyond a certain volume | High-stakes decisions (legal, medical, financial, public communications) |
| Structured output (enforced schema) | High for structured data, low for free-form generation | Low — native in modern APIs (JSON mode, tool use) | Data extraction, classification, form filling |
FAQ
Why do AI models hallucinate?
Hallucination is structural, not a bug. An LLM predicts the statistically most likely next word, with no native ability to verify its claims. Four mechanisms drive it: over-generalisation, interpolation between conflicting sources, pressure to answer rather than admit ignorance, and time lag (knowledge cutoff).
How can I detect an AI hallucination?
Three signals: citations that don't appear anywhere on external search, oddly precise numbers with no traceable source, and confident claims on recent or niche topics. Any output with legal, financial or medical stakes needs independent human verification — top models still hallucinate on 5 to 15 % of complex factual queries.
How do you reduce LLM hallucinations?
Three effective levers: RAG (Retrieval-Augmented Generation) to ground responses on source documents, explicit prompting to flag uncertainty, and systematic human review on high-stakes outputs. None eliminates the phenomenon; they cut its frequency and make errors detectable.
Who is legally liable for an AI hallucination?
The company deploying the chatbot or AI assistant remains liable for the statements produced — the Moffatt v. Air Canada ruling (February 2024) confirmed it: Air Canada was ordered to honour a refund policy invented by its own chatbot. The AI vendor is not party to the customer's claim.
Do all AI systems hallucinate?
All LLM-based generative models hallucinate, including the most recent. Deterministic systems (rule engines, full-text search) don't hallucinate but don't reason either. Hybrid architectures — RAG + LLM + verifiable citations — reduce the risk but don't remove it; hallucination is a property of the generative paradigm.
Is RAG enough to eliminate hallucinations?
No. RAG significantly reduces hallucinations by grounding answers on source documents, but the model can still rephrase, mix or distort retrieved passages. A sound setup requires explicit citation of source passages, careful chunking, and continuous evaluation of the grounding rate.
See also
Further reading
Moffatt v. Air Canada, Civil Resolution Tribunal of British Columbia, February 2024
Sources
- Moffatt v. Air Canada, Civil Resolution Tribunal of British Columbia, decision of February 14, 2024 (2024 BCCRT 149). https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html
- AI Incident Database, incident #563, Air Canada chatbot hallucination, 2024. https://incidentdatabase.ai/cite/563