When you connect a document to an AI chatbot, two things happen before it can answer a single question from it: chunking and embedding. Neither is complicated once you see what each is actually doing.
Chunking: breaking a document into pieces
A chatbot doesn't search your whole document at once — it searches small pieces of it. Chunking splits a document into sections small enough to be specific (so a search doesn't pull back an entire ten-page guide when the visitor asked about one setting) but large enough to still make sense on their own (so a chunk isn't half a sentence with no context).
Good chunk boundaries usually follow the natural structure of the document — headings, paragraphs, list items — rather than cutting at an arbitrary character count. This is also why well-structured docs produce better chatbot answers: see our post on writing docs AI chatbots answer well from.
Embeddings: turning text into something searchable by meaning
An embedding converts a chunk of text into a list of numbers — a vector — that represents its meaning. Two chunks about the same topic end up with vectors that are mathematically close together, even if they don't share many of the same words. That's what makes it possible to search by meaning instead of by matching keywords: a visitor asking "how do I get my money back" can retrieve a chunk titled "Refund Policy" even though none of those words overlap.
Every document you connect gets chunked and each chunk gets embedded, all before any visitor asks a question — this happens in the background when you upload a file or sync a source, not at chat time.
Why this two-step process matters
Chunking controls what gets compared. Embeddings control how the comparison happens. Get chunking wrong (chunks too big, too small, or split mid-thought) and even a perfect embedding model can't retrieve cleanly. Get embeddings wrong (or mix vectors from different models) and even perfectly chunked content won't match correctly.
Once chunks are embedded, finding the right one for a given question is a search problem — covered in how vector search actually finds the right answer.
Related reading
- What is RAG, explained for non-engineers — the bigger picture chunking and embeddings fit into.
- Grounded vs. hallucinated answers — why this pipeline is what keeps answers honest.
- 5 things everyone gets wrong about AI chatbots for support — including the myth that more data beats better structure.
- What does it actually cost to run an AI chatbot? — how the embedding step described here shows up on a bill.