The answer looked completely fine. That was the problem.
The question
A visitor asked our own chatbot, mid-testing, what the Pro plan cost per month. Simple question, answered a hundred times a day, backed by a pricing page we'd written ourselves. The chatbot answered instantly, cited a source, and gave a number that was wrong by ten dollars.
What actually happened
We'd updated pricing a few weeks earlier — and updated the live pricing page. What we hadn't done was delete the old internal draft page that still had the previous number sitting in a connected Notion workspace, technically outdated, never marked as deprecated, never unlinked. Two documents, same topic, contradicting numbers, both still indexed.
Vector search doesn't know one of two similar passages is "the real one" and the other is stale — it just finds what's closest in meaning to the question. Both pricing chunks were close. The retrieval step picked one, generation answered from it confidently, and nothing about the reply looked uncertain, because from the model's side, nothing was uncertain — it had a clean, specific passage to answer from. It just happened to be the wrong one.
Why this is scarier than an empty gap
A question with no answer in your docs is a known, visible failure — the chatbot says "I don't know," and that gets flagged and fixed. Two documents quietly disagreeing produces the worst kind of failure: a confident, cited, well-formed answer that's simply incorrect, with nothing in the reply itself hinting that anything's wrong. Grounding prevents hallucination; it does nothing to prevent grounding in the wrong source.
What we changed
We added a retrieval-confidence check: when the top two retrieved chunks for a question score close enough to each other on a sensitive topic like pricing, the system now hedges instead of picking one silently — either flags the conflict for review or asks a human to confirm before it goes out as a citation-backed fact. We also went back and cleaned up every workspace for exactly this pattern: old drafts, deprecated pages, anything answering the same question as something newer. Auto-sync keeps content current; it doesn't clean up the duplicates sitting alongside it, and that part's still on us.
The takeaway
We spend a lot of time talking about what happens when a chatbot has no source to answer from. This was a reminder that having too many sources — some of them stale — can be just as dangerous, and a lot harder to notice, because the failure mode doesn't look like failure. It looks like a normal, confident, correctly-formatted answer. Just the wrong one.
Related reading
- Grounded vs. hallucinated answers — why this failure mode slips past the usual honesty check.
- Auto-sync so your chatbot never goes stale — keeps content current, but doesn't remove what should have been deleted.
- We pointed Lumen Chat at our own docs — another round of testing this product on itself.