Same question, three different chatbot setups, three very different outcomes. Watching them side by side makes the difference between "an AI chatbot" and "a good AI chatbot" a lot less abstract.
The question: "If I'm on the annual plan and cancel after 3 months, do I get a partial refund?"
Setup 1: a general AI model with no connection to any documentation
"Most annual plans offer a prorated refund for unused months, so you should be able to get a partial refund for the remaining 9 months. I'd recommend checking with support to confirm the exact process."
This sounds completely reasonable — confident, specific-sounding, even helpful. It's also entirely made up. Nothing about "most annual plans" has anything to do with this company's actual refund policy, because the model was never given it. This is hallucination in its most dangerous form: not obviously wrong, just unearned confidence dressed up as an answer.
Setup 2: a chatbot grounded in documentation, but poorly chunked
"Our refund policy allows cancellations within the billing period. Annual plans are billed yearly."
This chatbot did retrieve something real — but the passages it pulled back were split too coarsely to isolate the actual answer to this question. It has true facts and no synthesis, which reads as evasive without being wrong. This is what happens when chunking is an afterthought: the right information exists in the source, but not in a retrievable shape.
Setup 3: a chatbot grounded in well-structured documentation
"Annual plans aren't prorated for early cancellation, but you do keep access through the end of the paid year rather than losing it immediately. If you'd like, I can start that cancellation for you now, or connect you with billing if you want to discuss options first."
Specific, correct, and it tells the visitor what happens next instead of stopping at the fact. This is what retrieval done well actually looks like — not a smarter model, the same model, handed a clean, well-scoped passage to answer from.
What actually changed between the three
Nothing about the model got smarter between setup 1 and setup 3. What changed was whether it had something real to answer from, and whether that source material was structured well enough to retrieve cleanly. That's the entire story behind most "our chatbot gives bad answers" complaints — it's almost never the model, and almost always what it was or wasn't given to work with.
Related reading
- Grounded vs. hallucinated answers — the mechanism setup 1 is missing entirely.
- Chunking and embeddings 101 — the mechanism setup 2 got wrong.
- Writing docs AI chatbots answer well from — how to get to setup 3 on your own content.
- Why more parameters doesn't always mean a better answer — why a bigger model wasn't the fix here either.