CrawlerDesk

AI crawler plumbing for any origin.

Managed llms.txt + 30-day crawler digests — not WordPress, not CF-dashboard-only.

Serve discovery files, keep 30 days of AI crawler path hits, and forward a client-ready weekly digest — for brand.com, docs, and SaaS origins agencies already manage.

Start agency seat — $49/mo See demo digest

No Shopify Liquid install. Email digests only in v0. Sign-up on app.crawlerdesk.com.

Agency $49/mo · 5 sites · client invites · white-label PDF Single site $19/mo Compare plans

Live on demo.crawlerdesk.com

Demo may show a labeled sample week — not your traffic.

What it is

Managed discovery files

Edit llms.txt / llms-full.txt in the desk; one body served to every client.

30-day crawler analytics

Path × known AI bot hits with CSV export — longer than typical free CF windows.

Client-ready digests

Weekly email + shareable /d/{token} link + agency white-label PDF on the $49 plan.

Who it’s for

Lead: GEO / SEO agencies running multi-client retainers who need a forwardable weekly proof sheet.

Also: Single-site docs / SaaS / brand.com operators ($19).

Not for: Citation share-of-voice buyers, Shopify Liquid “add llms.txt” shoppers, teams that only need the Cloudflare dashboard.

How it works

  1. Create an agency seat on app.crawlerdesk.com.
  2. Attach up to 5 client sites (invite clients with /i/{code}).
  3. Publish discovery files; crawler hits accumulate for 30 days.
  4. Forward the weekly /d link or white-label PDF — revoke anytime.

Pricing

Agency — recommended

$49/mo
  • Up to 5 sites
  • Client invites
  • White-label PDF digests
  • Shareable weekly digests
Start agency seat

Single site

$19/mo
  • One origin
  • Same plumbing: files, 30-day hits, weekly digest
Start single site
  • Unpaid: edits/digests stop immediately; files keep serving 15 days, then hard cut.
  • Custom hostname attach uses Cloudflare for SaaS (may show staged until zone ops complete).
  • Referral links are attribution only — no payouts.

Works beside GetIntel — not instead of it.

GetIntel drafts the fix; CrawlerDesk keeps it live and measurable.

GetIntel helps draft visibility fixes; CrawlerDesk keeps files live and measurable across weeks with a digest your client can open without logging in.

FAQ

Does this block AI bots at the edge?
No. We serve files and measure hits that reach CrawlerDesk. We do not claim off-Worker enforcement.
Shopify storefronts?
We don’t sell “add llms.txt to Shopify Liquid.” Use CrawlerDesk for brand.com / docs / SaaS origins you control.
Slack digests?
Not in v0 — email only.
Why not just Cloudflare AI Crawl Control?
Great if you’re deep in CF already; free metrics windows are short and multi-tenant white-label digests aren’t the product. CrawlerDesk is origin-agnostic plumbing + agency share.
Can clients pay separately?
On the agency plan, invited client sites bill to the agency seat within the 5-site limit.

Agency $49/mo · 5 sites · client invites — or single site $19/mo

Start agency seat — $49/mo Sign in