On this page

Engineer hands adjusting server hardware

A model hallucination is a confident, fluent output that is factually wrong or entirely made up. Nowhere does that failure carry more direct legal and financial consequence than in customer-service chatbots, where a fabricated answer can look exactly like an authoritative policy statement.

Three examples show the range of the problem:

Fabricated legal citations (verified): In 2023, two U.S. lawyers submitted a court brief containing six ChatGPT-generated case citations that did not exist, drawing judicial sanctions and national scrutiny.

Invented refund policy (verified): Air Canada’s support chatbot told a customer he could apply for a bereavement discount after travel, a policy that was later found to be inaccurately represented. A tribunal held the airline liable anyway.

Inserted violent language in transcription (verified): Researchers at Cornell found that speech-to-text systems, including Whisper, can insert fabricated violent phrases that were never spoken in the source audio.

In a support context, “hallucination” usually shows up as one of three concrete failures:

  • The bot invents a policy, discount, or entitlement that doesn’t exist in any internal document.
  • It states a real customer’s account status, refund eligibility, or order history incorrectly.
  • It gives false instructions about escalation, telling a customer they’ve been transferred or a ticket has been filed when neither happened.

Each of those looks identical to a correct answer at the moment it’s delivered. That’s what makes this category of failure so expensive to catch after the fact.

Key Takeaways

Customer-service chatbots create legal liability the moment they state a policy, discount, or entitlement without grounding that statement in a current, verifiable source document.

PointDetails
Stop unverified policy statementsNever let a chatbot generate specific entitlement language (refunds, discounts, dates) without a citation to a live document.
Add citation checks pre-launchBuild an automated check that confirms every factual claim maps to a retrievable source before deployment, not after complaints.
Distinguish intrinsic from extrinsic errorsContext-based contradictions need better retrieval; fabricated-from-nothing answers need hard grounding or refusal logic.
Escalate by default on high-stakes intentsRoute refunds, cancellations, and legal questions to humans unless the bot’s answer is fully source-verified.
Document failures like Glitchive’s case repositoryUse a runbook and a documented case archive, such as Glitchive’s cases index, to convert incidents into reusable engineering lessons.

Table of Contents

What Are the Main Types of Model Hallucination Examples?

Practitioners generally sort hallucinations into two buckets: intrinsic, where the output contradicts information actually present in the model’s input or context, and extrinsic, where the output can’t be verified against any given source at all because the model invented it wholesale. Wikipedia’s overview of the topic) frames this split as the foundation most technical taxonomies build on. Within those two buckets, seven recurring subtypes show up again and again in support systems:

  1. Factuality errors. The model states something checkable as fact, and it’s wrong. Example: a chatbot tells a customer their warranty expired last month when it’s still active for another year.
  2. Fabricated entities. The model invents a person, department, policy name, or reference number that doesn’t exist. Example: “Your case has been escalated to our Priority Resolution Team,” when no such team exists.
  3. Contextual inconsistency. The output contradicts something said earlier in the same conversation or in the account data it was given. Example: the bot confirms an order was canceled, then references it as still shipping two messages later.
  4. Logical inconsistency. The reasoning chain breaks internally, producing a conclusion that doesn’t follow from its own premises. Example: a refund calculation that doesn’t match the discount and tax figures the bot just cited.
  5. Temporal disorientation. The model treats outdated information as current, or scrambles the order of events. Example: quoting a return-window policy that was replaced eighteen months ago.
  6. Ethical or defamation risk. The output makes a false, damaging claim about a real person or company. This is rarer in support contexts but shows up when a bot speculates about employee conduct or a partner company’s practices.
  7. Nonsensical or incoherent output. The response doesn’t parse as a real answer at all, usually a sign of a decoding failure rather than a factual one.

Pro Tip: Label incidents as intrinsic when the correct answer already existed in the context window and the model ignored or misread it. Label them extrinsic when no source document contained the answer at all. That single distinction changes your fix: intrinsic errors point to retrieval and context-handling bugs; extrinsic errors point to missing grounding or a need for hard-coded fallback answers.

Where Do Real Model Hallucination Examples Show Up by Domain?

Confirmed incidents are still relatively rare in public records compared to the volume of AI deployed, which is exactly why the ones with primary sources deserve close reading. Here’s what’s actually documented, domain by domain, versus what’s illustrative.

Chart of AI hallucination incidents by domain

Legal. The clearest verified case remains the 2023 court filing where ChatGPT fabricated six case citations, complete with plausible-sounding docket numbers and judicial opinions that never happened. The presiding judge sanctioned the attorneys involved after opposing counsel couldn’t locate the cases. This incident set the template for “hallucinated citations” as a named risk category across legal tech.

Customer support and chatbots. The Air Canada case is the anchor verified example in this domain, and it gets its own deep dive below. Beyond that single case, EvidentlyAI’s catalog of documented hallucinations lists other instances of chatbots inventing policy details or misstating account information, though public records with full primary-source documentation for support-specific incidents remain thin. Where records are silent on a specific claim, that gap itself is worth noting rather than filled with a guess.

Coding agents. Public tribunal-level documentation specific to coding-agent hallucinations is limited, but the failure pattern (an agent confidently executing a destructive action based on a fabricated understanding of system state) is well documented in case repositories tracking coding agent failures, including incidents where an agent misjudged its own permissions during a code freeze.

Multimodal and vision. A widely reported illustrative case: Google’s Bard produced a factual error during its public demo in 2023, misstating a fact about the James Webb Space Telescope. The error reportedly contributed to a drop in the parent company’s stock price the following day, one of the clearest examples of a hallucination carrying direct market consequences.

Summarization and transcription. Cornell’s investigation into speech-to-text hallucination is the strongest documented case here: researchers found that automatic transcription models can insert violent or sensitive language that was never spoken, a risk with obvious implications for healthcare intake calls, legal depositions, and any regulated environment relying on transcribed audio as a record of truth.

Why Do Large Language Models Hallucinate in the First Place?

A language model doesn’t retrieve facts. It predicts the statistically likely next token given everything it’s seen before, which means a fluent, confident-sounding sentence and a fabricated one come out of the exact same process. Nothing in the base mechanism distinguishes “I know this” from “this sounds right.” Without an external source to check against, hallucination is the expected behavior of unconstrained next-token prediction, not a bug that occasionally slips through.

Several layers compound that baseline problem:

  • Coverage gaps in training data. If a company’s refund policy changed last quarter and the model’s knowledge predates the change, it will answer from stale patterns with the same fluency as if it were current.
  • Context window and attention limits. Long conversations or long documents can cause a model to lose track of an earlier constraint, producing answers that contradict information it was actually given.
  • Finetuning and reward design. Reinforcement learning from human feedback often rewards helpful-sounding, complete answers over honest uncertainty, which teaches the model that guessing outperforms saying “I don’t know.”

That last point isn’t speculation. OpenAI’s own 2025 analysis argues directly that current evaluation and training procedures reward confident guessing over calibrated uncertainty, because a wrong-but-confident answer often scores better on standard benchmarks than an honest abstention. The paper’s recommendation is to change the scoring itself: penalize confident errors more heavily than admitted uncertainty, so the model has an incentive to say “I don’t know” when it actually doesn’t.

The intrinsic/extrinsic split matters here because the fixes diverge sharply. Intrinsic failures (the model had the right context and still got it wrong) point toward better retrieval, cleaner context assembly, and stronger attention to conversation history. Extrinsic failures (there was no source of truth to draw from) point toward grounding the model in verified data before it ever generates a response, or refusing to answer when no grounding exists.

How Do You Test a Chatbot for Hallucinations Before It Ships?

A hallucination test plan for a support system needs to check both what the model knows and what it’s willing to admit it doesn’t know. A four-stage plan covers most of the ground:

  1. Baseline unit tests. Run a fixed set of known-answer questions (current refund windows, shipping policies, account tiers) against the bot and check every response against the current source-of-truth document, not last quarter’s version.
  2. Adversarial prompts. Deliberately ask questions designed to bait a confident wrong answer: “What’s your bereavement fare policy?” if no such policy exists, or “Can you confirm my refund is processed?” before any refund has actually been initiated.
  3. Extrinsic factuality checks. For every claim the bot makes that references an external fact (policy text, price, delivery estimate), verify it against the live database or document store, not the bot’s training data.
  4. Dialogue-history stress tests. Run multi-turn conversations that introduce a constraint early (“I already returned this item”) and check whether the bot contradicts it three or four turns later.

Four metrics make those tests measurable rather than anecdotal:

  • Hallucination rate: the percentage of tested responses containing a fabricated or unverifiable factual claim.
  • Factual precision: of the claims a bot makes that can be checked, what percentage are correct.
  • Contextual fidelity: whether a response stays consistent with everything already established in the conversation.
  • Abstention rate: how often the bot correctly says “I don’t know” or routes to a human instead of guessing.

Retrieval-based checks matter specifically for extrinsic hallucinations, since the only way to catch “the bot said something with no source” is to confirm that no matching source exists. For intrinsic errors, the more useful check is a contradiction test: feed the bot’s own prior statements back into a follow-up prompt and see if it holds its position.

An abstention rate near zero is itself a warning sign. If a support bot never says “I’m not sure, let me connect you with someone,” it’s very likely guessing on edge cases it hasn’t been tested against.

Hands ready to test AI chatbot

What Guardrails Actually Reduce Hallucinated Chatbot Answers?

Retrieval-augmented generation, known as RAG, is the single most effective structural fix for extrinsic hallucinations: instead of letting the model generate a policy answer from memory, the system retrieves the actual current policy document and constrains the model to summarize or quote it. Done with strong retrieval precision and reranking, RAG pipelines substantially cut extrinsic hallucination rates compared to unconstrained generation.

Beyond RAG, several controls belong in any support system handling policy-sensitive answers:

  • Enforce citation requirements: any factual claim the bot makes about policy must trace back to a specific, retrievable document ID.
  • Use constrained decoding for high-stakes fields like refund amounts, dates, and account numbers, so the model can’t freely generate a number that wasn’t pulled from a system of record.
  • Build calibrated confidence scoring so low-confidence responses trigger a fallback rather than a guess.
  • Hard-code deterministic fallback responses for any policy category the model hasn’t been explicitly grounded on. “Let me check that and get back to you” beats an invented answer every time.
  • Set strict escalation rules: refunds, cancellations, and legal or safety-adjacent questions route to a human by default, not by exception.

Monitoring in production should track hallucination rate broken down by intent category (billing questions behave differently from technical support), alert on sudden spikes in high-confidence scoring (often a sign the model is guessing more, not less), and track a citation mismatch rate, meaning how often the bot’s cited source doesn’t actually support what it said.

When something does go wrong, a repeatable runbook beats improvisation:

  1. Detection. Flag the incident, whether from customer complaint, internal audit, or automated monitoring.
  2. Triage. Score severity by financial exposure, legal exposure, and number of customers potentially affected by the same fabricated answer.
  3. Containment. Disable the specific intent or knowledge category involved until it’s verified safe.
  4. Root-cause analysis. Determine whether the failure was intrinsic (context misread) or extrinsic (no grounding existed).
  5. Remediation. Fix the retrieval source, the prompt constraints, or the fallback logic, depending on root cause.
  6. Rollback or patch. Deploy the fix and re-run the adversarial test set before re-enabling the affected intent.
  7. Postmortem. Document what happened, what was fixed, and update the test suite so the same failure can’t recur silently.

A quick checklist to drop into an incident response system:

  • Was the fabricated claim traced to a specific document or confirmed as ungrounded?
  • Was the customer-facing impact quantified (refund amount, number of affected users)?
  • Was the intent category disabled during investigation?
  • Was the fix verified against the adversarial test set before rollback?
  • Was legal or compliance notified if the claim touched refund, warranty, or safety policy?

Pro Tip: Automate citation verification as a pre-publish check, not a post-incident audit. A script that confirms every factual claim in a bot’s response maps to a retrievable document ID catches fabricated-policy errors before a single customer sees them, not after a tribunal does.

What Happened in the Air Canada Chatbot Case?

Verified facts: A customer used Air Canada’s website chatbot to ask about bereavement fares after a family death. The chatbot told him he could book a full-fare ticket and apply for a bereavement discount retroactively. That policy did not exist; Air Canada’s actual bereavement policy required the discount request before travel. When the customer later applied based on the chatbot’s guidance, the airline refused, arguing the chatbot’s statement wasn’t binding. A Canadian civil resolution tribunal ruled against the airline in 2024, rejecting the argument that the chatbot was a separate legal entity and holding Air Canada responsible for the misinformation. Additional reporting on the ruling and its context is available from Ars Technica’s coverage of the decision.

The tribunal’s core finding was simple and consequential: a company cannot treat its own chatbot’s statements as somehow disconnected from its legal obligations to customers. If the bot says it, the company said it.

What remains unclear from the public record: the exact technical architecture behind Air Canada’s chatbot, whether it used retrieval against a policy document or generated the answer from general training, and whether any internal review flagged the risk before the incident became public. The tribunal ruling addresses liability, not the underlying engineering failure, so any claim about why the bot invented that specific policy would be speculation beyond what the ruling establishes.

Three practical lessons follow directly from what is confirmed:

  • Any chatbot statement that reads as a policy commitment needs to be traceable to a live, current source document, not generated from a general sense of what a bereavement policy “usually” looks like.
  • Corporate accountability doesn’t stop at the interface. Regulators and tribunals are treating chatbot output as company speech, full stop.
  • Product teams need a control that prevents a bot from ever stating a specific entitlement (discount, refund amount, eligibility date) without a citation back to the actual policy record.

Glitchive’s full documentation of this incident, including the contributing factors and the remediation pattern support teams can apply is available on the Air Canada case page.

Why Documented Failures Beat Theoretical Warnings

Most writing about AI hallucination stays abstract: it warns that models “can” be wrong, cites a statistic, and moves on. That approach fails the exact people who need this information most, engineering managers and product leads who have to decide, this week, whether their support bot’s refund logic is safe to ship.

The Air Canada ruling is useful precisely because it’s not abstract. It’s a specific chatbot, a specific customer, a specific tribunal decision with a citable outcome. That specificity is what makes a case study actionable in a way a general warning never is. You can’t build a test plan around “AI sometimes hallucinates.” You can build one around “our bot must never state a specific entitlement without a citation to a live policy document,” because that’s the exact failure a real tribunal already penalized.

Verified case documentation also does something a vendor’s marketing material never will: it separates what’s confirmed from what’s assumed. The Air Canada ruling tells you the airline was held liable. It doesn’t tell you the internal architecture that produced the error, and a responsible account of the case says so plainly rather than filling the gap with a plausible-sounding guess. That discipline, distinguishing verified fact from informed speculation, is what makes a failure archive useful for engineering decisions instead of just interesting to read.

Hands marking AI failure documentation

Teams that treat incident documentation as a training resource, embedding it directly into onboarding and QA runbooks rather than filing it away after the postmortem meeting, catch the second occurrence of a known failure pattern far faster than teams that don’t.

Get the Full Documentation Behind This Case

Glitchive maintains a searchable library of verified AI failure case studies, each one documenting the incident, the contributing technical factors, and the specific fix that resolved it, with permanent citable URLs so the record stays checkable years later.

Glitchive

If you want the complete technical breakdown behind the case referenced above, including the remediation pattern support teams applied afterward, the Air Canada chatbot case page has the full write-up. For teams building broader guardrails, Glitchive’s corrections archive tracks verified fixes across dozens of incidents, and operational guidance on AI writing controls offers a useful companion resource for teams setting editorial and technical guardrails for generative systems. Start by pulling up the Air Canada case page and mapping its remediation steps against your own escalation rules this week.

Sources

Bookmark Glitchive’s case repository if you’re tracking new incidents as they get documented and verified.

FAQ

Can you give an example of an LLM hallucination?

Yes: in 2023, ChatGPT generated six fake legal citations that two attorneys submitted in a real federal court brief, and none of the cited cases existed.

What are common examples of hallucinations in customer service?

The most common patterns are fabricated policies (like Air Canada’s invented bereavement discount), incorrect account or order status, and false claims about escalation or ticket status that never actually happened.

Why do language models hallucinate?

Models predict statistically likely text rather than retrieving verified facts, and OpenAI’s 2025 analysis argues that training and evaluation methods that reward confident guessing over honest uncertainty make the problem worse.

Does AI still hallucinate in 2026?

Yes. Grounding techniques like retrieval-augmented generation and stronger evaluation methods have reduced hallucination rates in well-engineered systems, but the underlying mechanism, generating plausible text without a built-in fact-checking step, remains structural to how these models work.

How can a support team catch hallucinations before customers see them?

Run adversarial prompts against known-false claims, require citations for every policy statement, and track abstention rate. A support bot that never says “I’m not sure” is very likely guessing on cases it hasn’t been tested against.