How is this different from Google Analytics?

Google Analytics shows you traffic. Shadow shows you traffic, AI bot activity, what AI platforms say about your brand, AND tells you what to do about all of it. It's analytics + AI intelligence + action steps in one tool.

Do I need to install anything?

For basic monitoring (bot detection, AI perception, readiness score) — nope, just enter your URL. For full visitor analytics (clicks, behavior, sessions), add one script tag. One-click integrations for Vercel, Shopify, WordPress, and more.

Will it slow down my site?

No. The script is under 5KB and loads async. Zero impact on page speed or Core Web Vitals. External monitoring has literally no impact — it watches from the outside.

What AI bots does Shadow detect?

All of them. GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot, and dozens more. The Shadow Network means new bots get identified across all users instantly.

What do you mean by "actionable steps"?

Shadow doesn't just show you graphs. It says things like: "ChatGPT has your pricing wrong — add structured data to /pricing to fix it" or "Your bounce rate on /features is 68% — here's why and what to change." Specific, do-it-today recommendations.

Can Shadow block bots?

Shadow is a telescope, not a shield. It shows you who's visiting and what AI says about you. It generates block rules and robots.txt configs you can apply — but it doesn't intercept traffic.

Yes. Shadow never collects PII. IP addresses are hashed after classification. No cookies on your visitors. All Shadow Network data is anonymized. GDPR compliant by design.

How do I block AppleBot-Extended?

Add this to your robots.txt file: User-agent: AppleBot-Extended / Disallow: /

Does AppleBot-Extended respect robots.txt?

Yes, AppleBot-Extended by Apple respects robots.txt directives.

AI Training Respects robots.txt

AppleBot-Extended

Name: Open Shadow
Author: Henry Barouni

by Apple First seen: 2024-06

About

Apple's AI training crawler, separate from the standard Applebot used for Siri and Spotlight. AppleBot-Extended collects data specifically for training Apple's foundation models and Apple Intelligence features.

Purpose

AI model training for Apple Intelligence

User Agent String

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0; compatible; Applebot-Extended/0.3)

How to Control in robots.txt

🚫 Block AppleBot-Extended

User-agent: AppleBot-Extended
Disallow: /

✅ Allow AppleBot-Extended

User-agent: AppleBot-Extended
Allow: /

Complete Guide: How to Block AppleBot-Extended

Server-level blocking, nginx configs, Cloudflare rules, Next.js middleware, and more →

Other Bots by Apple

Applebot

Siri answers, Spotlight search, and Safari web suggestions

AI Assistant

Is AppleBot-Extended crawling your site?

Run a free scan to check if Apple's crawler is accessing your website.

Check if AppleBot-Extended is crawling YOUR site →