How to Block AI Bots on Slim 4 (PHP): Complete 2026 Guide

Layer 1: robots.txt

Place robots.txt in public/ — Slim 4's document root (where index.php lives). Your web server serves files from this directory directly, before PHP runs.

# public/robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: Applebot-Extended
User-agent: PerplexityBot
User-agent: Diffbot
User-agent: cohere-ai
User-agent: FacebookBot
User-agent: omgili
User-agent: omgilibot
User-agent: Amazonbot
User-agent: DeepSeekBot
User-agent: MistralBot
User-agent: xAI-Bot
User-agent: AI2Bot
Disallow: /

Alternatively, define a Slim route for dynamic generation:

$app->get('/robots.txt', function (Request $request, Response $response) {
    $body = <<<ROBOTS
User-agent: *
Allow: /

User-agent: GPTBot
Disallow: /
ROBOTS;
    $response->getBody()->write($body);
    return $response->withHeader('Content-Type', 'text/plain');
});

Layer 2: noai meta tag

Add the meta tag to your PHP or Twig base template. Pass an override variable from controllers that need different behaviour:

<!-- app/Views/layout.php -->
<meta name="robots" content="<?= htmlspecialchars($robots ?? 'noai, noimageai') ?>">

{# templates/base.html.twig #}
<meta name="robots" content="{{ robots|default('noai, noimageai') }}">

// Allow indexing on a specific public page
return $this->view->render($response, 'page.html.twig', [
    'robots' => 'index, follow',
]);

Layers 3 & 4: PSR-15 middleware

Implement Psr\Http\Server\MiddlewareInterface with a single process() method. Return a new response to block. Call $handler->handle($request) to pass through.

src/Middleware/AiBotBlocker.php

<?php

declare(strict_types=1);

namespace App\Middleware;

use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\MiddlewareInterface;
use Psr\Http\Server\RequestHandlerInterface;
use Slim\Psr7\Response;

class AiBotBlocker implements MiddlewareInterface
{
    private const AI_BOT_PATTERNS = [
        'gptbot', 'chatgpt-user', 'oai-searchbot',
        'claudebot', 'anthropic-ai', 'claude-web',
        'google-extended', 'ccbot', 'bytespider',
        'applebot-extended', 'perplexitybot', 'diffbot',
        'cohere-ai', 'facebookbot', 'meta-externalagent',
        'omgili', 'omgilibot', 'amazonbot',
        'deepseekbot', 'mistralbot', 'xai-bot', 'ai2-bot',
    ];

    private const EXEMPT_PATHS = [
        '/robots.txt',
        '/sitemap.xml',
        '/favicon.ico',
    ];

    public function process(
        ServerRequestInterface $request,
        RequestHandlerInterface $handler
    ): ResponseInterface {
        $path = $request->getUri()->getPath();

        // Always pass through exempt paths
        if (in_array($path, self::EXEMPT_PATHS, true)) {
            return $handler->handle($request);
        }

        $ua = strtolower($request->getHeaderLine('User-Agent'));

        foreach (self::AI_BOT_PATTERNS as $pattern) {
            if (str_contains($ua, $pattern)) {
                // Layer 4: hard 403 — do NOT call $handler->handle()
                $response = new Response(403);
                $response->getBody()->write('Forbidden');
                return $response->withHeader('Content-Type', 'text/plain');
            }
        }

        // Layer 3: X-Robots-Tag on legitimate responses
        // PSR-7 is immutable — withAddedHeader() returns a NEW instance
        $response = $handler->handle($request);
        return $response->withAddedHeader('X-Robots-Tag', 'noai, noimageai');
    }
}

Registering the middleware

<?php
// public/index.php (or app bootstrap)

use App\Middleware\AiBotBlocker;
use Slim\Factory\AppFactory;

$app = AppFactory::create();

// Global — runs on every request
// Add last (outermost) so it intercepts before other middleware
$app->addMiddleware(new AiBotBlocker());

// Slim runs middleware in LIFO order for the response phase.
// Add bot blocker after (above) other middleware like auth and body parsing.
$app->addBodyParsingMiddleware();

$app->run();

Route-scoped blocking

To protect only API routes and leave frontend routes unaffected:

use Slim\Routing\RouteCollectorProxy;

// Single route
$app->get('/api/products', ProductHandler::class)->add(new AiBotBlocker());

// Route group — all /api/* routes protected
$app->group('/api', function (RouteCollectorProxy $group) {
    $group->get('/products', ProductHandler::class);
    $group->get('/orders', OrderHandler::class);
    $group->post('/cart', CartHandler::class);
})->add(new AiBotBlocker());

Route-level middleware runs in LIFO order — middleware added closest to the route runs first. Add the bot blocker outermost (last) to intercept before auth or body parsing.

Verification

# Layer 1 — robots.txt
curl https://your-slim-app.com/robots.txt

# Layer 3 — X-Robots-Tag on legitimate request
curl -I https://your-slim-app.com/api/products
# Expected: X-Robots-Tag: noai, noimageai

# Layer 4 — hard block on bot user-agent
curl -A "Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)" \
  -I https://your-slim-app.com/api/products
# Expected: HTTP/1.1 403 Forbidden

# robots.txt must pass through even for bot UAs
curl -A "GPTBot" -I https://your-slim-app.com/robots.txt
# Expected: HTTP/1.1 200 OK

Slim 3 vs Slim 4

If you are upgrading from Slim 3, the middleware signature is different:

function ($request, $response, $next) {
    // ... check UA ...
    if ($isBot) {
        return $response->withStatus(403);
    }
    $response = $next($request, $response);
    return $response->withHeader('X-Robots-Tag', 'noai, noimageai');
}

public function process(
    ServerRequestInterface $request,
    RequestHandlerInterface $handler
): ResponseInterface {
    // ... check UA ...
    if ($isBot) {
        return new Response(403); // No $handler->handle()
    }
    $response = $handler->handle($request);
    return $response->withAddedHeader('X-Robots-Tag', 'noai, noimageai');
}

FAQ