How is this different from Google Analytics?

Google Analytics shows you traffic. Shadow shows you traffic, AI bot activity, what AI platforms say about your brand, AND tells you what to do about all of it. It's analytics + AI intelligence + action steps in one tool.

Do I need to install anything?

For basic monitoring (bot detection, AI perception, readiness score) — nope, just enter your URL. For full visitor analytics (clicks, behavior, sessions), add one script tag. One-click integrations for Vercel, Shopify, WordPress, and more.

Will it slow down my site?

No. The script is under 5KB and loads async. Zero impact on page speed or Core Web Vitals. External monitoring has literally no impact — it watches from the outside.

What AI bots does Shadow detect?

All of them. GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot, and dozens more. The Shadow Network means new bots get identified across all users instantly.

What do you mean by "actionable steps"?

Shadow doesn't just show you graphs. It says things like: "ChatGPT has your pricing wrong — add structured data to /pricing to fix it" or "Your bounce rate on /features is 68% — here's why and what to change." Specific, do-it-today recommendations.

Can Shadow block bots?

Shadow is a telescope, not a shield. It shows you who's visiting and what AI says about you. It generates block rules and robots.txt configs you can apply — but it doesn't intercept traffic.

Yes. Shadow never collects PII. IP addresses are hashed after classification. No cookies on your visitors. All Shadow Network data is anonymized. GDPR compliant by design.

What is Google-NotebookLM and what does it do?

Google-NotebookLM is a web crawler operated by Google to power NotebookLM — Google's AI-powered research and note-taking tool. When a NotebookLM user adds a URL as a source, the Google-NotebookLM crawler fetches and reads that page so the AI can answer questions, generate summaries, create study guides, and even produce AI-generated podcast episodes ('Audio Overviews') from the content. It crawls on demand — triggered by individual user actions, not by bulk background indexing.

What is NotebookLM and why has it gone viral?

NotebookLM is Google's AI-powered research assistant that lets users create 'notebooks' by adding sources (URLs, PDFs, Google Docs, YouTube videos) and then have an AI answer questions about those specific sources. It went viral in late 2024 primarily because of its 'Audio Overviews' feature — which generates remarkably realistic two-person podcast conversations summarizing your uploaded content. Researchers, students, journalists, and knowledge workers use it heavily to synthesize information from multiple sources.

How is Google-NotebookLM different from Google-Extended and Gemini-Deep-Research?

Google operates several distinct AI crawlers, each with a separate purpose: Google-Extended collects training data to improve Gemini models — it's a bulk background crawler. Gemini-Deep-Research crawls pages when Gemini Advanced users run autonomous research queries. Google-NotebookLM crawls pages when a specific NotebookLM user adds a URL as a source — it's user-triggered and tied to a specific notebook session. All three are separate bots with separate robots.txt directives. Blocking one has no effect on the others.

Does blocking Google-NotebookLM affect my Google Search rankings?

No. Google-NotebookLM is not Googlebot. Blocking it via robots.txt has zero effect on your search indexing, rankings, or appearance in Google Search results. Googlebot is the only Google crawler you must never block if you care about search visibility.

Why should I care if NotebookLM reads my pages?

Several legitimate reasons: (1) Paywalled content — if NotebookLM reads your full premium content, subscribers can share it with the AI, bypassing your paywall. (2) Copyright — your content may be used to generate derivative AI works (summaries, podcasts) without attribution or compensation. (3) Zero-click — NotebookLM users interact with an AI version of your content inside Google, not your actual site. You lose the visit, the ad revenue, and the conversion opportunity. (4) Principle — you may simply not want your work used as AI training material or source material without consent.

Does Google-NotebookLM respect robots.txt?

Yes. Google documents Google-NotebookLM as a distinct crawler and it is designed to respect robots.txt directives. Google's documented commitment to robots.txt compliance across its crawler family applies to Google-NotebookLM. For paywalled content specifically, Google also recommends using subscriber-specific access controls rather than relying solely on robots.txt.

What is the Google-NotebookLM user agent string?

The robots.txt token is: Google-NotebookLM. The full observed user agent string is: Mozilla/5.0 (compatible; Google-NotebookLM/1.0; +https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers). In robots.txt, use: User-agent: Google-NotebookLM followed by Disallow: /.

GoogleRespects robots.txtAI Notebook

How to Block Google-NotebookLM: Stop Google's AI Research Tool from Crawling Your Site

NotebookLM went viral for its AI-generated podcasts. Behind it sits a crawler that reads your pages when users add your URLs as sources. Here's how to block it without touching your Google Search rankings.

Updated March 2026

What Is NotebookLM?

NotebookLM is Google's AI-powered research assistant. Users create "notebooks" by adding sources — URLs, PDFs, Google Docs, YouTube videos — and then interact with an AI that can answer questions, generate summaries, create study guides, and produce remarkably realistic two-person AI podcast episodes ("Audio Overviews") based on those sources.

It went viral in late 2024 primarily because of Audio Overviews — the feature generates podcast-style conversations that discuss and debate the content of uploaded sources with uncanny realism. Researchers, students, journalists, and knowledge workers adopted it rapidly.

When a user adds a URL as a source, Google-NotebookLM fetches and reads that page. The crawler is triggered by individual user actions — it doesn't bulk-crawl the web. Each fetch is tied to a specific notebook session.

Why Publishers Should Pay Attention

NotebookLM creates derivative works from your content: summaries, study guides, Q&A, and audio podcasts. Users interact with an AI version of your content inside Google's product — not your website. You lose:

-The page visit (zero-click)

-Ad revenue from that visit

-Conversion opportunities (signups, subscriptions)

-Control over how your content is presented and discussed

-Attribution (the AI podcast doesn't link to your site)

How to Block Google-NotebookLM

Add this to your robots.txt:

robots.txtBlock NotebookLM only

User-agent: Google-NotebookLM
Disallow: /

To block all Google AI crawlers while keeping search rankings:

robots.txtBlock all Google AI, keep search

# Keep search indexing
User-agent: Googlebot
Allow: /

# Block AI training
User-agent: Google-Extended
Disallow: /

# Block Deep Research
User-agent: Gemini-Deep-Research
Disallow: /

# Block NotebookLM
User-agent: Google-NotebookLM
Disallow: /

Google Search is unaffected

Google-NotebookLM is completely separate from Googlebot. Blocking it has zero effect on your Google Search rankings, indexing, or organic traffic.

The Audio Overviews Concern

NotebookLM's Audio Overviews feature generates AI podcast episodes from your content. Two synthetic voices discuss, debate, and explain your material in a conversational format. These audio files can be shared and distributed.

For publishers, this is a new category of derivative work. Your original article or research becomes an AI-generated podcast that:

No attribution required

The generated podcast doesn't necessarily credit your original work or link to your site.

Shareable output

Users can share the generated audio with anyone — distributing a derivative of your content outside your site.

Competes with your format

If you produce podcasts, research summaries, or educational content, AI-generated versions of your own content compete directly with you.

What Blocking Does (and Doesn't) Do

What it stops

• NotebookLM from reading your URLs when users add them
• AI-generated summaries and podcasts from your content
• Your content being used as a notebook source

What it doesn't stop

• Users uploading PDFs or copy-pasting your content manually
• Google-Extended training crawls (block separately)
• Deep Research crawls (block separately)
• Google Search rankings (unaffected)

Frequently Asked Questions

Can users still add my content to NotebookLM if I block the crawler?

They can still upload PDFs, paste text, or add Google Docs containing your content. The robots.txt block only prevents the Google-NotebookLM crawler from fetching URLs. It doesn't prevent manual uploads — that's a user action, not a crawl.

Is NotebookLM content used for Gemini model training?

Google has stated that NotebookLM content is not used to train Gemini models. The Google-NotebookLM crawler retrieves content for the user's notebook session only. However, for training opt-out, you should also block Google-Extended.

How many people use NotebookLM?

NotebookLM saw rapid adoption after Audio Overviews went viral in late 2024. Google hasn't published exact user numbers, but it's widely used in education, journalism, research, and professional knowledge work. The user base is growing as Google integrates it deeper into its product ecosystem.

What happens when a user tries to add my blocked URL?

If you block Google-NotebookLM via robots.txt, the crawler cannot fetch your page. NotebookLM will likely show an error indicating the source could not be loaded. The user may then resort to manual methods (copy-paste, PDF upload) which the block cannot prevent.

Related Guides

How to Block Google-Extended

Google's AI training crawler

How to Block Gemini Deep Research

Google's deep research crawler

How to Block Applebot-Extended

Apple's AI training crawler

robots.txt for AI Bots (Complete Guide)

51+ crawlers, full reference table

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.