bubblio

BubblioBot

BubblioBot is the crawler behind Bubblio's knowledge feature: a website owner adds their own site in the Bubblio dashboard, BubblioBot reads its public pages, and their AI support character can then answer visitors' questions from that content. It identifies itself on every request as:

BubblioBot/1.0 (+https://bubblio.dev/bot)

What it does — and doesn't

  • Opt-in only. It crawls a site only after that site's owner explicitly adds the URL in their Bubblio dashboard. It never discovers or wanders onto third-party sites — crawls stay on the same site, including through redirects.
  • Public pages only. Plain GET requests, no cookies, no logins, no form submissions — it can only see what any logged-out visitor (or search engine) sees.
  • robots.txt first. It fetches and honors robots.txt before anything else (matching User-agent: BubblioBot, else *), and skips pages marked noindex. If robots.txt can't be read, it doesn't crawl.
  • Polite. About one request per second, sitemap-first, roughly 200 pages per site, 2 MB per page, and a re-visit about once a week (or when the owner requests a refresh).

What happens to the content

  • Indexed content powers only the site owner's own assistant — it is never shared across Bubblio customers and never used to train AI models.
  • When the owner removes a source (or their account), its indexed content is deleted immediately.
  • Pages that disappear from a site, or that add noindex, are dropped from the index on the next crawl.

Blocking or limiting BubblioBot

Add a rule to your site's robots.txt — it takes effect on the next crawl:

# Block BubblioBot entirely
User-agent: BubblioBot
Disallow: /

# …or keep it out of specific paths only
User-agent: BubblioBot
Disallow: /private/

Questions

Seeing unexpected traffic from this user-agent, or have a question about how your content is handled? Email bubblio@andresio.com and include your domain — we'll respond quickly.