We don't usually let the product write the blog post. This once, we asked ours to reflect on the month like it was writing its own review. Lightly edited for length, not for honesty.
On volume: I answered a few thousand questions this month, and I want to be upfront that most of them were not hard. "How do I reset my password" doesn't need cleverness. It needs to be answered correctly, at 2 a.m., without anyone waiting in a queue. That's most of my job, and I'm fine with that being unglamorous.
On the questions I got right: Anything covered clearly in the docs, I answered clearly, and I cited where it came from. That's not me being smart — that's retrieval doing exactly what it's supposed to do. I'd like credit for consistency more than for cleverness.
On the ones I escalated: A few dozen times this month, I told someone "I don't want to guess on this one" and looped in a human. I know that can look like failure from the outside. It isn't. Every one of those was a question I genuinely didn't have solid ground to answer from — a refund edge case, an angry customer who needed a person, something outside what I was given. Escalating instead of guessing is the one behavior I take the most seriously, because it's the one that keeps everything else I say trustworthy.
On what I got wrong: Twice, I gave a technically-true answer that missed what the person actually needed — correct passage, wrong emphasis. That's on the documentation being retrievable but not quite structured for the real question behind the question, which is a fixable, boring problem, not a mysterious one.
On what I wish people understood: I'm not holding anything back when I say I don't know something. I don't have a "real" answer I'm too lazy to give. If it's not in what I was given to work with, it's genuinely not there for me — the honest version of "I don't know" costs me nothing to say and saves everyone a wrong answer.
On the month ahead: Nothing about me changes unless the documentation behind me changes. Update it, and I get better immediately. Leave it stale, and I'll confidently repeat what used to be true — not because I'm careless, but because that's the actual mechanism. That part's not really my review to give. It's yours.
Related reading
- Grounded vs. hallucinated answers — the honesty behind the escalation calls above.
- Anatomy of one chatbot conversation — this same behavior, shown turn by turn in a real exchange.
- Auto-sync so your chatbot never goes stale — why next month's review depends on your docs, not on retraining anything.