How to Block AI Bots on Medusa.js: Complete 2026 Guide

Layer 1: robots.txt

Place robots.txt in the static/ directory at your Medusa project root. Medusa serves this directory at the web root automatically — no route definition required.

# static/robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: Applebot-Extended
User-agent: PerplexityBot
User-agent: Diffbot
User-agent: cohere-ai
User-agent: FacebookBot
User-agent: omgili
User-agent: omgilibot
User-agent: Amazonbot
User-agent: DeepSeekBot
User-agent: MistralBot
User-agent: xAI-Bot
User-agent: AI2Bot
Disallow: /

Project structure: static/robots.txt lives alongside src/, medusa-config.ts, and package.json at the root of your Medusa project.

Layer 2: noai meta tag

Medusa is a headless backend — it returns JSON, not HTML. Add the noai meta tag in your storefront. The official Medusa Next.js Starter uses the App Router:

// app/layout.tsx
export const metadata = {
  // ...existing metadata
  other: { robots: 'noai, noimageai' },
};

// nuxt.config.ts
export default defineNuxtConfig({
  app: { head: { meta: [
    { name: 'robots', content: 'noai, noimageai' }
  ]}}
})

<!-- src/routes/+layout.svelte -->
<svelte:head>
  <meta name="robots" content="noai, noimageai">
</svelte:head>

Layers 3 & 4: defineMiddlewares()

Medusa v2 discovers middleware from a single file: src/api/middlewares.ts. The file must export a default function named defineMiddlewares() that returns { routes: [...] }. Each route entry has a matcher (string glob or RegExp) and a middlewares array of Express-compatible functions.

src/api/middlewares.ts

import type { MedusaNextFunction, MedusaRequest, MedusaResponse } from '@medusajs/medusa';

const AI_BOT_PATTERNS = [
  'gptbot', 'chatgpt-user', 'oai-searchbot',
  'claudebot', 'anthropic-ai', 'claude-web',
  'google-extended', 'ccbot', 'bytespider',
  'applebot-extended', 'perplexitybot', 'diffbot',
  'cohere-ai', 'facebookbot', 'meta-externalagent',
  'omgili', 'omgilibot', 'amazonbot',
  'deepseekbot', 'mistralbot', 'xai-bot', 'ai2-bot',
];

const EXEMPT_PATHS = new Set(['/robots.txt', '/sitemap.xml', '/favicon.ico']);

function aiBotBlocker(
  req: MedusaRequest,
  res: MedusaResponse,
  next: MedusaNextFunction
): void {
  const path = req.path.split('?')[0];

  // Always pass through exempt paths
  if (EXEMPT_PATHS.has(path)) {
    next();
    return;
  }

  const ua = (req.headers['user-agent'] || '').toLowerCase();

  // Layer 4: hard 403 block
  if (AI_BOT_PATTERNS.some(pattern => ua.includes(pattern))) {
    res.sendStatus(403);
    return;
  }

  // Layer 3: X-Robots-Tag — inject on all legitimate responses
  res.set('X-Robots-Tag', 'noai, noimageai');
  next();
}

export default defineMiddlewares({
  routes: [
    {
      matcher: '/**',
      middlewares: [aiBotBlocker],
    },
  ],
});

Storefront-only blocking

To block bots only on storefront API routes and leave admin and webhook routes unaffected:

export default defineMiddlewares({
  routes: [
    {
      matcher: '/store/**',
      middlewares: [aiBotBlocker],
    },
    // Webhooks: never block (payment providers use programmatic UA strings)
    // Admin: served separately — apply blocking at proxy/CDN level
  ],
});

Merging with existing middleware

If you already have a src/api/middlewares.ts (e.g. for CORS or authentication), add the bot blocker to the existing routes array. Order matters — middleware runs in array order. Place the bot blocker first so blocked requests never reach CORS or auth:

import { authenticate } from '@medusajs/medusa';
import type { MedusaNextFunction, MedusaRequest, MedusaResponse } from '@medusajs/medusa';
// ... aiBotBlocker function as above ...

export default defineMiddlewares({
  routes: [
    // 1. Bot blocking runs first — rejected requests never reach auth
    {
      matcher: '/store/**',
      middlewares: [aiBotBlocker],
    },
    // 2. Existing auth middleware for protected admin routes
    {
      matcher: '/admin/**',
      middlewares: [authenticate()],
    },
  ],
});

Verification

# Default Medusa backend port is 9000

# Layer 1 — robots.txt (must return 200 to all bots)
curl http://localhost:9000/robots.txt

# Layer 3 — X-Robots-Tag on legitimate store request
curl -I http://localhost:9000/store/products
# Expected: X-Robots-Tag: noai, noimageai

# Layer 4 — hard block on bot user-agent
curl -A "Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)" \
  -I http://localhost:9000/store/products
# Expected: HTTP/1.1 403 Forbidden

# robots.txt must stay accessible even to bots
curl -A "GPTBot" -I http://localhost:9000/robots.txt
# Expected: HTTP/1.1 200 OK

FAQ