How to Block AI Bots on Strapi: Complete 2026 Guide

Methods at a glance

1. robots.txt — public/ directory

Strapi uses koa-static to serve files from the public/ directory at the root URL. Place robots.txt there and it will be served at /robots.txt — no config needed.

The robots.txt for your Strapi instance should address the admin panel and API — not your public website content (that lives in your frontend's robots.txt):

# public/robots.txt — in your Strapi project root
# This file is for your Strapi backend domain (e.g. api.example.com or cms.example.com)
# Your frontend (example.com) has its own robots.txt — see your frontend framework guide

# Block AI training bots from the admin panel
User-agent: GPTBot
Disallow: /admin
Disallow: /api

User-agent: ChatGPT-User
Disallow: /admin
Disallow: /api

User-agent: ClaudeBot
Disallow: /admin
Disallow: /api

User-agent: Claude-Web
Disallow: /admin
Disallow: /api

User-agent: anthropic-ai
Disallow: /admin
Disallow: /api

User-agent: Google-Extended
Disallow: /admin
Disallow: /api

User-agent: Bytespider
Disallow: /admin
Disallow: /api

User-agent: CCBot
Disallow: /admin
Disallow: /api

User-agent: PerplexityBot
Disallow: /admin
Disallow: /api

# Block all bots from admin — no bot should index the CMS panel
User-agent: *
Disallow: /admin

2. Hard 403 blocking — custom Koa middleware

Strapi's middleware system wraps Koa. Create a middleware file in src/middlewares/ and register it in config/middlewares.ts. The middleware runs on every request — exempt admin, API, GraphQL, health-check, and static files before checking User-Agent.

src/middlewares/block-ai-bots.ts

// src/middlewares/block-ai-bots.ts
import { Strapi } from "@strapi/strapi";

// Compiled once at module load — not per-request
const BLOCKED_UAS =
  /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended|meta-externalagent|Diffbot|ImagesiftBot/i;

// Paths that must never be blocked
const EXEMPT_PREFIXES = ["/admin", "/api", "/graphql", "/_health", "/robots.txt"];

export default (_config: unknown, { strapi: _strapi }: { strapi: Strapi }) => {
  return async (ctx: any, next: () => Promise<void>) => {
    const path = ctx.path;

    // Always pass through exempt paths
    if (EXEMPT_PREFIXES.some((prefix) => path.startsWith(prefix))) {
      return await next();
    }

    const ua: string = ctx.get("user-agent") ?? "";

    if (BLOCKED_UAS.test(ua)) {
      ctx.status = 403;
      ctx.body = "Forbidden";
      return; // Do not call next() — request ends here
    }

    await next();
  };
};

config/middlewares.ts — register the middleware

// config/middlewares.ts
export default [
  "strapi::logger",
  "strapi::errors",
  "strapi::security",
  "strapi::cors",
  "strapi::poweredBy",
  "strapi::query",
  "strapi::body",
  "strapi::session",
  "strapi::favicon",
  "strapi::public",          // serves public/ directory — keep before bot blocker
  {
    name: "global::block-ai-bots",
    config: {},
  },
];

3. X-Robots-Tag on API responses

X-Robots-Tag is a response header. On JSON API responses it acts as a signal to bots that respect headers — some do check it before indexing API content. Add it in a separate Strapi middleware or combine it with the bot-blocking middleware.

// src/middlewares/x-robots-tag.ts
export default () => {
  return async (ctx: any, next: () => Promise<void>) => {
    await next();

    // Set X-Robots-Tag on all non-admin responses
    if (!ctx.path.startsWith("/admin")) {
      ctx.set("X-Robots-Tag", "noai, noimageai");
    }
  };
};

// config/middlewares.ts — add after the bot blocker
export default [
  // ... other middlewares
  { name: "global::block-ai-bots", config: {} },
  { name: "global::x-robots-tag", config: {} },
];

4. Protecting content from direct API scraping

If your Strapi API is public (no authentication required), any bot can query your content directly — bypassing your frontend's robots.txt and meta tags entirely. The definitive fix is Strapi's built-in Roles & Permissions.

Option A — Require authentication on sensitive endpoints

In the Strapi admin panel: Settings → Roles → Public. Uncheck find and findOne for any collection you want to protect. Only authenticated requests will be able to query those endpoints.

Option B — Block known AI bot User-Agents at /api/* in middleware

Less robust (bots can fake User-Agents) but appropriate as a second layer if your API must stay public. Update the middleware to optionally block bots from API routes:

// src/middlewares/block-ai-bots.ts — with optional API blocking
const BLOCKED_UAS =
  /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended/i;

export default (config: { blockApi?: boolean }, _ctx: any) => {
  return async (ctx: any, next: () => Promise<void>) => {
    // Always pass admin panel through
    if (ctx.path.startsWith("/admin")) return await next();

    // Optionally exempt /api (set blockApi: true in config to also block API routes)
    if (!config.blockApi && ctx.path.startsWith("/api")) return await next();

    if (ctx.path === "/robots.txt") return await next();

    const ua: string = ctx.get("user-agent") ?? "";
    if (BLOCKED_UAS.test(ua)) {
      ctx.status = 403;
      ctx.body = "Forbidden";
      return;
    }

    await next();
  };
};

// config/middlewares.ts
export default [
  // ...
  {
    name: "global::block-ai-bots",
    config: { blockApi: true }, // set true to also block known AI bots from /api/*
  },
];

5. noai meta tags — your frontend

Strapi does not render HTML pages for end users — your frontend does. Add noai meta tags in your frontend framework. Here are the guides for the most common Strapi frontend pairings:

If Strapi and your frontend run on the same domain (e.g., Strapi serving custom pages via custom routes), add the meta tag via a Strapi custom controller that renders HTML, or use a custom page template. This is uncommon — most Strapi setups are API-only.

6. nginx reverse proxy

Putting nginx in front of Strapi adds a network-level blocking layer before Node.js even sees the request. More efficient for high-traffic APIs — nginx handles the 403 response without loading the Node process.

# /etc/nginx/sites-available/strapi
map $http_user_agent $blocked_bot {
  default           0;
  "~*GPTBot"        1;
  "~*ChatGPT-User"  1;
  "~*OAI-SearchBot" 1;
  "~*ClaudeBot"     1;
  "~*Claude-Web"    1;
  "~*anthropic-ai"  1;
  "~*Google-Extended" 1;
  "~*Bytespider"    1;
  "~*CCBot"         1;
  "~*PerplexityBot" 1;
}

server {
  listen 80;
  server_name api.example.com;

  # Never block robots.txt
  location = /robots.txt {
    proxy_pass http://localhost:1337;
  }

  # Never block the admin panel from legitimate users
  location /admin {
    proxy_pass http://localhost:1337;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }

  # Block AI bots from public routes and API
  location / {
    if ($blocked_bot) {
      return 403 "Forbidden";
    }

    proxy_pass http://localhost:1337;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

7. Deployment

Docker — multi-stage build

# Dockerfile
FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build   # npx strapi build

FROM node:22-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/build ./build
COPY --from=builder /app/config ./config
COPY --from=builder /app/public ./public
COPY --from=builder /app/src ./src
RUN npm ci --omit=dev

EXPOSE 1337
CMD ["npm", "start"]   # npx strapi start

# Mount a Docker volume for /app/public/uploads to persist media uploads

Frequently asked questions