Skip to content
Guides/Symfony

How to Block AI Bots on Symfony: Complete 2026 Guide

Symfony is the leading enterprise PHP framework — powering Drupal, Magento, and Laravel under the hood. Bot blocking uses Symfony's EventSubscriber system: a single PHP class that listens on two kernel events — KernelEvents::REQUEST for hard 403 blocking before any controller runs, and KernelEvents::RESPONSE for injecting the X-Robots-Tag header on every outgoing response. No extra packages required.

Four protection layers

1
robots.txt — Place in public/ — Symfony's web root, served automatically by any web server
2
noai meta tag — {{ robots|default("noai, noimageai") }} in your Twig base template
3
X-Robots-Tag header — KernelEvents::RESPONSE subscriber — added to every response before it leaves PHP
4
Hard 403 block — KernelEvents::REQUEST subscriber at priority 9999 — fires before routing and controllers

Layer 1: robots.txt

Place robots.txt directly in the public/ directory — Symfony's document root. Any web server (Apache, Nginx, Caddy) serves it as a static file before PHP is even invoked.

# public/robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: Applebot-Extended
User-agent: PerplexityBot
User-agent: Diffbot
User-agent: cohere-ai
User-agent: FacebookBot
User-agent: omgili
User-agent: omgilibot
User-agent: Amazonbot
User-agent: DeepSeekBot
User-agent: MistralBot
User-agent: xAI-Bot
User-agent: AI2Bot
Disallow: /

Symfony's default public/ already contains index.php. Add robots.txt alongside it — no route definition or controller needed. Committed to version control so it deploys with your app.

Layer 2: noai meta tag

Add the tag to your Twig base template using the built-in default filter. Per-page override is handled by passing a robots variable from the controller.

Base Twig template

{# templates/base.html.twig #}
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>{% block title %}My App{% endblock %}</title>

    {# AI bot training opt-out. Per-page override: pass robots from controller. #}
    <meta name="robots" content="{{ robots|default('noai, noimageai') }}">

    {% block stylesheets %}{% endblock %}
</head>
<body>
    {% block body %}{% endblock %}
    {% block javascripts %}{% endblock %}
</body>
</html>

Controller — default (no override)

<?php
// src/Controller/HomeController.php
namespace App\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

class HomeController extends AbstractController
{
    #[Route('/', name: 'home')]
    public function index(): Response
    {
        // No robots key → base template defaults to "noai, noimageai"
        return $this->render('home/index.html.twig');
    }
}

Controller — per-page override

<?php
// Public pages that should be indexed normally:
#[Route('/about', name: 'about')]
public function about(): Response
{
    return $this->render('pages/about.html.twig', [
        'robots' => 'index, follow',
    ]);
}

Layers 3 & 4: EventSubscriber

A single EventSubscriber class listens on two kernel events. KernelEvents::REQUEST fires first (before routing) — used for the hard 403 block. KernelEvents::RESPONSE fires last — used to inject the X-Robots-Tag header on all responses.

<?php
// src/EventSubscriber/AiBotSubscriber.php
namespace App\EventSubscriber;

use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\Event\RequestEvent;
use Symfony\Component\HttpKernel\Event\ResponseEvent;
use Symfony\Component\HttpKernel\KernelEvents;

class AiBotSubscriber implements EventSubscriberInterface
{
    // Case-insensitive substrings — matches partial UA strings
    private const AI_BOT_PATTERNS = [
        'gptbot', 'chatgpt-user', 'oai-searchbot',
        'claudebot', 'anthropic-ai', 'claude-web',
        'google-extended', 'ccbot', 'bytespider',
        'applebot-extended', 'perplexitybot', 'diffbot',
        'cohere-ai', 'facebookbot', 'meta-externalagent',
        'omgili', 'omgilibot', 'amazonbot',
        'deepseekbot', 'mistralbot', 'xai-bot', 'ai2-bot',
    ];

    // Always allow — crawlers must be able to read robots.txt
    private const EXEMPT_PATHS = [
        '/robots.txt',
        '/sitemap.xml',
        '/favicon.ico',
    ];

    public static function getSubscribedEvents(): array
    {
        return [
            // Layer 4: run before routing (priority 9999 > controller priority 0)
            KernelEvents::REQUEST  => ['onKernelRequest',  9999],
            // Layer 3: add header to every response
            KernelEvents::RESPONSE => ['onKernelResponse', 0],
        ];
    }

    public function onKernelRequest(RequestEvent $event): void
    {
        // Only act on the main request, not sub-requests (e.g. Twig includes)
        if (!$event->isMainRequest()) {
            return;
        }

        $request = $event->getRequest();
        $path    = $request->getPathInfo();

        // Exempt robots.txt, sitemap, favicon — must be readable by all bots
        if (in_array($path, self::EXEMPT_PATHS, true)) {
            return;
        }

        $ua = strtolower($request->headers->get('User-Agent', ''));

        foreach (self::AI_BOT_PATTERNS as $pattern) {
            if (str_contains($ua, $pattern)) {
                // Layer 4: hard block — short-circuit before any controller runs
                $event->setResponse(new Response('Forbidden', Response::HTTP_FORBIDDEN, [
                    'Content-Type' => 'text/plain',
                ]));
                return;
            }
        }
    }

    public function onKernelResponse(ResponseEvent $event): void
    {
        if (!$event->isMainRequest()) {
            return;
        }

        // Layer 3: inject X-Robots-Tag on every response
        $event->getResponse()->headers->set('X-Robots-Tag', 'noai, noimageai');
    }
}

Key points

  • isMainRequest() guards against sub-requests (e.g. Twig render() calls, ESI fragments) — without it, the bot check would fire multiple times per page.
  • Priority 9999 on REQUEST ensures the subscriber runs before Symfony's Router (priority 32) and Security (priority 8). Controllers run at priority 0.
  • $event→setResponse() short-circuits the kernel — no further REQUEST listeners fire and no controller is invoked.
  • str_contains() is PHP 8.0+. For PHP 7.x, use strpos($ua, $pattern) !== false.
  • RESPONSE listener runs after the controller — it adds the header to the already-built response, including error responses and redirects.

Service registration

With autoconfigure: true (Symfony default since 3.4), any class implementing EventSubscriberInterface is automatically tagged and registered. No manual configuration needed.

Automatic (autoconfigure)

# config/services.yaml
services:
    _defaults:
        autowire: true
        autoconfigure: true  # ← automatically tags EventSubscriberInterface implementations
        public: false

    App\:
        resource: '../src/'
        exclude:
            - '../src/DependencyInjection/'
            - '../src/Entity/'
            - '../src/Kernel.php'

Manual (if autoconfigure is off)

# config/services.yaml
services:
    App\EventSubscriber\AiBotSubscriber:
        tags:
            - { name: kernel.event_subscriber }

Environment-specific disabling

To disable the subscriber in the dev environment only:

# config/services.yaml
when@dev:
    services:
        App\EventSubscriber\AiBotSubscriber:
            autoconfigure: false
            tags: []

Verification

# Layer 1 — robots.txt served as static file
curl -I https://yoursite.com/robots.txt

# Layer 3 — X-Robots-Tag header on a real page
curl -I https://yoursite.com/

# Layer 4 — hard 403 when GPTBot user-agent is sent
curl -A "GPTBot" -I https://yoursite.com/
# Expected: HTTP/1.1 403 Forbidden

# Verify robots.txt exempt from hard block
curl -A "GPTBot" -I https://yoursite.com/robots.txt
# Expected: HTTP/1.1 200 OK

Unit testing the subscriber

<?php
// tests/EventSubscriber/AiBotSubscriberTest.php
use App\EventSubscriber\AiBotSubscriber;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\Event\RequestEvent;
use Symfony\Component\HttpKernel\HttpKernelInterface;

class AiBotSubscriberTest extends TestCase
{
    private function makeEvent(string $ua, string $path = '/'): RequestEvent
    {
        $kernel  = $this->createMock(HttpKernelInterface::class);
        $request = Request::create($path);
        $request->headers->set('User-Agent', $ua);
        return new RequestEvent($kernel, $request, HttpKernelInterface::MAIN_REQUEST);
    }

    public function testGptBotIsBlocked(): void
    {
        $event = $this->makeEvent('Mozilla/5.0 (compatible; GPTBot/1.0)');
        (new AiBotSubscriber())->onKernelRequest($event);
        $this->assertEquals(403, $event->getResponse()->getStatusCode());
    }

    public function testRobotsTxtIsExempt(): void
    {
        $event = $this->makeEvent('GPTBot', '/robots.txt');
        (new AiBotSubscriber())->onKernelRequest($event);
        $this->assertNull($event->getResponse()); // No response set → request proceeds normally
    }

    public function testLegitimateUserPassesThrough(): void
    {
        $event = $this->makeEvent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)');
        (new AiBotSubscriber())->onKernelRequest($event);
        $this->assertNull($event->getResponse());
    }
}

FAQ

Should I use KernelEvents::REQUEST or a Symfony firewall for AI bot blocking?

Use KernelEvents::REQUEST in an EventSubscriber. The Security firewall is designed for authentication — it requires user providers, tokens, and passport logic that add complexity with no benefit for simple user-agent matching. The EventSubscriber approach is framework-idiomatic, requires no extra configuration, and is easy to unit test without standing up a full kernel.

What priority should I use for the KernelEvents::REQUEST listener?

Use 9999. Symfony dispatches REQUEST listeners from highest to lowest priority. Controllers run at priority 0. The Router listener runs at priority 32. Priority 9999 ensures the bot check fires before routing, security checks, and any controller logic — blocked requests never touch your database or business logic.

Do I need to manually tag the EventSubscriber as kernel.event_subscriber?

No — with autoconfigure: true (the Symfony default since 3.4), any class implementing EventSubscriberInterface is automatically tagged. Symfony reads getSubscribedEvents() at compile time and registers the listeners. If you see no effect after creating the subscriber, run php bin/console debug:event-dispatcher kernel.request to verify it appears in the listener list.

How do I override the noai meta tag per page in Twig?

Pass a robots variable from your controller: return $this->render("page.html.twig", ["robots" => "index, follow"]). In your Twig base template use {{ robots|default("noai, noimageai") }}. The default filter supplies the fallback when robots is absent from the template context — no Twig extension required.

Will this EventSubscriber affect Symfony's profiler or debug toolbar?

No — the Symfony Profiler runs at the /_profiler path and uses internal sub-requests. The isMainRequest() check prevents the subscriber from acting on sub-requests. The Profiler's own user-agent does not match any AI bot pattern. If you test with a custom user-agent in devtools that accidentally matches a pattern, disable the subscriber in the dev environment using when@dev in services.yaml.

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.