Why chatbots make things up, and what it costs you
Large language models generate text by predicting what plausibly comes next; they have no built-in mechanism for checking truth. When a customer asks about a policy the model has never seen, it will often produce a confident, fluent, wrong answer rather than admit ignorance. That confidence is precisely what makes hallucination dangerous in a customer-facing setting.
The costs are real. In a widely reported 2024 case, a Canadian tribunal ordered Air Canada to honour a bereavement discount its website chatbot had invented, rejecting the airline's argument that the bot was responsible for its own statements. In the UK, misleading statements to consumers can create legal exposure under consumer protection law, and beyond the law there is the slower cost: refunds honoured, complaints handled and trust eroded, one wrong answer at a time. If your bot speaks for your business, your business owns what it says.
Grounding: force answers from your own content
The single most effective guardrail is retrieval-augmented generation (RAG). Instead of letting the model answer from its general training, the system first searches your approved knowledge base (policies, product pages, help articles), then instructs the model to answer only from the retrieved passages. The model becomes a fluent interface to your documents rather than a source of its own claims.
- Instruct explicitly: answer only from the provided context; if it is not there, say so
- Show sources: link the article or policy each answer came from, so customers and staff can verify
- Keep the knowledge base current; a bot grounded in last year's price list hallucinates by proxy
- Date-stamp and version documents so stale content is easy to spot and retire
Need a hand with this?
Our team delivers AI Chatbots for UK businesses — with a free initial consultation, transparent fixed quotes and no lock-in contracts. Tell us what you're working on →
Refusal rules: teach the bot to say 'I don't know'
A customer-facing bot needs a written scope, and the model needs to be told what sits outside it. Define the topics the bot must never freelance on: binding price promises, legal or medical advice, claims about competitors, anything contractual. For each, give the model an explicit alternative behaviour, usually a polite redirect to a human.
Just as important, reward abstention. Models default to being helpful, so unless the prompt makes clear that 'I don't know, let me connect you with the team' is a good answer, the bot will guess instead. Write the refusal phrasing yourself so it stays on-brand, and test it: a refusal rule you have never seen fire is a rule you cannot trust.
- List prohibited topics explicitly in the system prompt, with the required fallback response
- Never let the bot invent discounts, delivery dates or stock levels; these must come from live data or not at all
- Test refusals deliberately by asking out-of-scope questions before launch
Confidence thresholds and output checks
Grounding and refusal rules live in the prompt; the next layer lives in code. When the retrieval step returns nothing sufficiently relevant, do not let the model answer at all: below a set relevance threshold, route straight to the fallback. This one check removes a large share of hallucinations, because the model never gets the chance to improvise around missing information.
Then validate what comes out. Structured output features in modern APIs let you force replies into a defined format, which makes them checkable by ordinary business rules before the customer sees them: prices matched against the live price list, links checked against an allow-list of your own domains, discount figures capped at authorised levels. For high-risk intents such as cancellations or complaints, a second model pass that verifies the draft answer against the retrieved sources is a cheap extra safety net.
Design the human handoff before launch
Escalation is not an admission of failure; it is the guardrail of last resort, and it should be designed with the same care as the happy path. Decide what triggers a handoff, what the human receives, and what happens when nobody is available.
- Trigger on low retrieval confidence, repeated failed answers, and keywords such as complaint, refund, cancel or legal
- Trigger on frustration signals: exclamation-heavy messages, repetition, explicit requests for a person
- Pass the full transcript and the customer's details to the agent so nobody repeats themselves
- Outside business hours, offer a callback or email capture rather than letting the bot soldier on
- Label the bot as a bot; pretending it is human destroys trust the first time it slips
Key Takeaway
Never rely on the model alone to be truthful. Layer four guardrails: ground every answer in your own documents through retrieval, write explicit refusal rules so 'I don't know' beats a confident guess, enforce confidence thresholds and business-rule validation in code, and design human escalation before launch. Then regression-test with a red-team question bank and review real transcripts weekly, because your business legally and reputationally owns whatever the bot says.
Test, monitor, repeat
Before launch, build a red-team question bank: the awkward, ambiguous and adversarial questions real customers will ask, including attempts to extract discounts or make the bot contradict policy. Write the correct answer, or correct refusal, for each, and run the set against every prompt change. Treat prompts like code: versioned, reviewed and regression-tested.
After launch, sample transcripts weekly, add thumbs-up/down feedback to every answer, and track escalation and resolution rates. Every hallucination found in the wild becomes a new test case. Guardrails are a practice, not a feature you switch on. If you want a customer-facing bot built with these layers from day one, our team can help.