How is this different from Google Analytics?

Google Analytics shows you traffic. Shadow shows you traffic, AI bot activity, what AI platforms say about your brand, AND tells you what to do about all of it. It's analytics + AI intelligence + action steps in one tool.

Do I need to install anything?

For basic monitoring (bot detection, AI perception, readiness score) — nope, just enter your URL. For full visitor analytics (clicks, behavior, sessions), add one script tag. One-click integrations for Vercel, Shopify, WordPress, and more.

Will it slow down my site?

No. The script is under 5KB and loads async. Zero impact on page speed or Core Web Vitals. External monitoring has literally no impact — it watches from the outside.

What AI bots does Shadow detect?

All of them. GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot, and dozens more. The Shadow Network means new bots get identified across all users instantly.

What do you mean by "actionable steps"?

Shadow doesn't just show you graphs. It says things like: "ChatGPT has your pricing wrong — add structured data to /pricing to fix it" or "Your bounce rate on /features is 68% — here's why and what to change." Specific, do-it-today recommendations.

Can Shadow block bots?

Shadow is a telescope, not a shield. It shows you who's visiting and what AI says about you. It generates block rules and robots.txt configs you can apply — but it doesn't intercept traffic.

Yes. Shadow never collects PII. IP addresses are hashed after classification. No cookies on your visitors. All Shadow Network data is anonymized. GDPR compliant by design.

What is the difference between meta-externalagent and facebookexternalhit?

facebookexternalhit is Meta's long-standing link preview crawler — it fetches pages when someone shares a URL on Facebook or Instagram so Meta can generate a title, description, and thumbnail. meta-externalagent is a separate, newer crawler that Meta uses specifically for AI training data collection for Llama and other models. Blocking facebookexternalhit removes link preview generation; blocking meta-externalagent blocks AI training crawls. They operate independently.

Does Meta respect robots.txt for meta-externalagent?

Meta has stated that meta-externalagent respects robots.txt Disallow directives. Unlike Bytespider, which has been documented ignoring robots.txt, meta-externalagent is generally considered compliant. A robots.txt block is sufficient for most publishers.

Will blocking meta-externalagent affect my Facebook link previews?

No. Facebook link previews are generated by facebookexternalhit, not meta-externalagent. Blocking meta-externalagent only stops Meta's AI training crawler. Your link previews on Facebook, Instagram, Messenger, and WhatsApp will continue to work normally.

What models is meta-externalagent used to train?

Meta uses web crawl data as a training source for its Llama family of open-source models (Llama 2, Llama 3, and future versions) as well as research models used internally. Meta has not published a detailed breakdown of which specific models use web crawl data versus licensed or synthetic datasets.

What user agent does meta-externalagent use?

Meta's AI training crawler uses 'meta-externalagent' as the robots.txt user agent token. The full user agent string is: meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler). Note that an earlier version used 'meta-externalagent/1.0'.

How to Block meta-externalagent: Stop Meta AI Training on Your Site (2026)

Related Guides

How to Block GPTBot

OpenAI's training crawler

How to Block ClaudeBot

Anthropic's training crawler

How to Block Bytespider

ByteDance's aggressive crawler

robots.txt for AI Bots (Complete Guide)

51+ crawlers, full reference table

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.

Scan My Site Free →