Most AI support tools look identical on a pricing page — "connect your docs," "go live in minutes," "reduce ticket volume." The differences that actually matter show up once you ask specific questions. Here are ten worth asking before you commit.
1. Does it answer from my content, or from general AI knowledge?
If a vendor can't clearly explain that answers are retrieved from your connected documentation specifically — not a general-purpose model's training data — that's the first red flag. See what RAG is and why it matters.
2. What does it do when it doesn't know the answer?
A tool that always produces a confident-sounding answer, even to questions your docs don't cover, will eventually hand a customer a wrong answer with total confidence. Ask specifically how it handles gaps.
3. Can it connect to where my docs already live?
If the only onboarding path is manually pasting content in, most teams never finish setup. Ask about direct connectors — Notion, Google Drive, website crawling — versus a one-time import.
4. How does content stay current?
Docs change. Ask whether the tool re-syncs automatically or requires a manual re-upload every time something changes — this is the difference between a chatbot that stays accurate and one that quietly goes stale.
5. How does escalation to a human actually work?
Ask what triggers a handoff, what context the human receives, and where the alert goes — a chatbot with no real escalation path will eventually strand a customer with a problem it can't solve.
6. Is my data isolated from other customers?
If the vendor serves multiple organizations, ask directly how retrieval is scoped — your chatbot should only ever be able to surface your own connected content, never another customer's.
7. What happens to my data if I connect Notion or Drive?
Ask what scope of access is requested (a specific folder or database, versus your entire account) and what's stored versus just referenced. A tool that can't answer this clearly hasn't thought about it enough.
8. How is pricing structured as we grow?
Message-based or conversation-based pricing tends to track actual usage; seat-based pricing built for human-agent tools often doesn't map cleanly onto a chatbot's actual cost driver.
9. How long until it's actually live?
Ask for the real number, not "minutes to set up" — connecting a source, customizing a widget, and confirming it appears correctly on your site is a fair estimate of true time-to-value.
10. Can I see an answer's source?
If the tool can't show which document an answer was pulled from, you have no way to audit or correct wrong answers at the root — you're stuck fixing symptoms instead of the actual gap in your docs.
None of these questions require a technical background to ask, and the answers usually tell you more than any feature comparison chart. See how Lumen Chat answers all ten if you want a concrete reference point while you evaluate.