Not every team keeps documentation in Notion or Drive. If your help center, FAQ, or product pages are just public pages on your website, Lumen Chat can crawl them directly — no export step required.
1. Give it a starting URL
Add a Website source and enter a seed URL — your help center's root page, for example. Lumen Chat follows links from that page within your domain, up to a page limit based on your plan, so you don't need to list every URL by hand.
2. JavaScript-rendered pages
Most static help center pages crawl cleanly out of the box. If your site renders content client-side (a React or Vue-driven docs site, for instance), enable JS-rendered page support so the crawler executes the page before reading it, rather than seeing an empty shell.
3. Re-crawling as your site changes
A website source can be re-synced the same way as any other — manually, or on an auto-sync schedule — so a published changelog entry or updated pricing page gets picked up without you re-adding anything.
When to use this instead of Notion or Drive
If your source of truth is public web pages rather than an internal workspace, crawling is usually the least-friction option: there's no OAuth flow, no folder or database picker — just a URL. If you later move that content into Notion or Drive, you can add those as additional sources without removing this one.
Related reading
- Turning your docs into an AI chatbot — how a crawled source fits into the overall setup.
- Setting up auto-sync so your chatbot never goes stale — keeping a crawled site current without re-crawling manually.
- Connecting Notion as a chatbot source — a good complement if part of your content lives outside your website.
Point Lumen Chat at your site and it'll do the crawling for you.