It's one thing to be told a model reads text as tokens instead of words. It's another to actually watch it happen. Take one ordinary customer question and walk through roughly how it gets broken apart before the model ever "sees" it as a sentence.
The sentence
"Can I get a refund if I cancel mid-month?"
Nine words, forty characters, one obvious meaning to a person. To a model, it's closer to this rough split:
Can I get a refund if I cancel mid - month ?
That's roughly a dozen tokens for a sentence a person reads in under two seconds — because the model isn't splitting on words, it's splitting on frequently-occurring text fragments, which is why "mid-month" breaks into three separate pieces while "refund" stays whole. Common whole words tend to survive as single tokens; less common combinations, hyphenated words, and anything the model saw less often during training tend to fragment. (See tokens and context windows, explained without the jargon for what a token actually is and why this matters for cost.)
Why the exact split is unpredictable from the outside
Where the cuts land depends entirely on what patterns showed up most often in the data a specific model was trained on — which is also part of why different models can read the same prompt and produce different answers. Two models can tokenize the identical sentence into a different number of pieces, using a completely different set of boundaries, and neither is more "correct" than the other.
Why this is more than trivia
Every one of those dozen-ish tokens counts against the model's context window budget before a single word of the conversation history or your documentation gets added. A short, plain-sounding question already has a real token cost attached to it — and a visitor's rambling, run-on question with unusual phrasing can quietly cost two or three times as many tokens as the same question asked plainly, before the model has produced a single word in return.
The takeaway
The model never reads "Can I get a refund if I cancel mid-month?" the way you just did. It reads a sequence of a dozen-ish fragments and predicts, one at a time, what fragment is most likely to come next — a process invisible from the outside, but one that shapes cost, speed, and even which model gives which answer.
Related reading
- Tokens and context windows, explained without the jargon — the concept behind the fragments shown above.
- Why two AI models can read the same prompt and disagree — how tokenization differences contribute to model-to-model variation.
- 300 milliseconds: everything that happens before a chatbot answers — where this step fits in the full request timeline.