Chatbot invented a refund policy; tribunal held the airline liable
← Back to the full caseRetrieval-grounded policy answers with inline citations and refusal outside coverage
- By AFL Editorial
ROOT CAUSE ADDRESSED
The gap between what the bot asserts and what the canonical policy corpus actually says — ungrounded answer synthesis with no consistency check against the authoritative source the answer itself linked to.
Fix
When a support bot can only repeat what the policy corpus says, a fabricated 90-day refund window is unproducible — no supporting passage exists, so no answer containing it can be assembled. That is the design: the canonical policy pages (bereavement terms, fare rules, refund conditions) are indexed into a retrieval layer; each question pulls the governing passages, and the reply is constrained to statements entailed by them, quoting or tightly paraphrasing the source with an inline citation to the exact page. Before display, a verification step — an entailment model, or rule-based matching for high-stakes terms like deadlines, amounts, and eligibility — compares every factual claim in the drafted answer against the retrieved passages and blocks the reply on contradiction or no-support. Where retrieval finds nothing on point, or the checker fails, the bot says so and hands off to a human agent rather than composing an answer. Applied to this incident, the second control matters as much as the first: the contradiction with the linked "Bereavement travel" page is precisely the comparison the tribunal faulted Air Canada for never making.
Prevention
Regenerate the retrieval index automatically from the live policy pages so bot answers cannot lag policy edits, and gate deploys on a policy Q&A eval suite whose expected answers are derived from the same corpus. Track refusal and contradiction-block rates in production as leading indicators of coverage gaps.
Tradeoffs
A team whose policy pages are themselves stale, contradictory, or ambiguous gains little here: grounding faithfully reproduces bad source text, so corpus hygiene has to come first. Real engineering cost and per-answer latency are the price of the verification step, and a low-traffic support widget may be safer and cheaper as a curated FAQ or decision tree with no generation at all. Aggressive refusal thresholds trade user frustration for safety, and questions that hinge on account-specific data the corpus does not contain get no help from any of this.
Other solutions on this case
Retrieval-grounded policy answers with inline citations and refusal outside coverage
This is the only solution currently on file.