Skip to content
Guides/Vapor (Swift)

How to Block AI Bots on Vapor (Swift): Complete 2026 Guide

Vapor is the most widely used server-side Swift web framework — async/await-native, built on SwiftNIO, and deployable on Linux and macOS. Bot blocking uses Vapor's AsyncMiddleware protocol: a single struct that intercepts every request, optionally blocks it, and sets response headers before returning.

Four protection layers

1
robots.txt — Place in Public/ — FileMiddleware serves it automatically, or define a direct route
2
noai meta tag — <meta name="robots" content="#(robots ?? \"noai, noimageai\")"> in your Leaf base template
3
X-Robots-Tag header — response.headers.add() after awaiting the next responder in AsyncMiddleware
4
Hard 403 block — AI bot regex check in AsyncMiddleware — return Response(status: .forbidden) without calling next

Layer 1: robots.txt

Two options: static file via FileMiddleware, or a direct route. The static approach is simpler; the direct route works in Docker environments where the Public/ directory may not be bundled.

Option A: Public/ directory (FileMiddleware)

Create the file and register FileMiddleware in configure.swift. Vapor serves everything in Public/ automatically.

# Public/robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: Applebot-Extended
User-agent: PerplexityBot
User-agent: Diffbot
User-agent: cohere-ai
User-agent: FacebookBot
User-agent: omgili
User-agent: omgilibot
User-agent: Amazonbot
Disallow: /
// Sources/App/configure.swift
import Vapor

public func configure(_ app: Application) async throws {
    // Serve Public/ directory at the web root
    app.middleware.use(FileMiddleware(publicDirectory: app.directory.publicDirectory))

    // Register bot-blocking middleware AFTER FileMiddleware
    // so /robots.txt is served before the bot check fires
    app.middleware.use(AiBotMiddleware())

    try routes(app)
}

Option B: Direct route

Register a route before attaching the global middleware. Required for compiled Linux binaries where Public/ may not exist on the filesystem.

// Sources/App/routes.swift
let ROBOTS_TXT = """
User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
Disallow: /
"""

func routes(_ app: Application) throws {
    // Register before bot middleware is applied
    app.get("robots.txt") { req -> Response in
        var headers = HTTPHeaders()
        headers.contentType = .plainText
        return Response(status: .ok, headers: headers, body: .init(string: ROBOTS_TXT))
    }
}

Layer 2: noai meta tag

Add the tag to your Leaf base template. The ?? operator provides a default when the variable is absent from the render context.

Base Leaf template

{{-- Resources/Views/base.leaf --}}
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>#(title ?? "My App")</title>

  {{-- AI bot training opt-out. Per-page override: pass robots key in render context. --}}
  <meta name="robots" content="#(robots ?? "noai, noimageai")">
</head>
<body>
  #import("body")
</body>
</html>

Controller — default (no override)

// No robots key → base template defaults to "noai, noimageai"
func index(req: Request) async throws -> View {
    return try await req.view.render("index", ["title": "Home"])
}

Controller — per-page override

// Public pages that should be indexed normally:
func about(req: Request) async throws -> View {
    return try await req.view.render("about", [
        "title": "About",
        "robots": "index, follow",
    ])
}

Layers 3 & 4: AsyncMiddleware

A single AsyncMiddleware implementation handles both the X-Robots-Tag header and the hard 403 block. It intercepts every request before your route handlers execute.

// Sources/App/Middleware/AiBotMiddleware.swift
import Vapor

struct AiBotMiddleware: AsyncMiddleware {

    // Case-insensitive substring matching — same pattern as all platforms
    private static let aiBotPatterns: [String] = [
        "gptbot", "chatgpt-user", "oai-searchbot",
        "claudebot", "anthropic-ai", "claude-web",
        "google-extended", "ccbot", "bytespider",
        "applebot-extended", "perplexitybot", "diffbot",
        "cohere-ai", "facebookbot", "meta-externalagent",
        "omgili", "omgilibot", "amazonbot",
        "deepseekbot", "mistralbot", "xai-bot", "ai2-bot",
    ]

    // Always allow — crawlers must be able to read robots.txt
    private static let exemptPaths: Set<String> = [
        "/robots.txt", "/sitemap.xml", "/favicon.ico",
    ]

    func respond(
        to request: Request,
        chainingTo next: AsyncResponder
    ) async throws -> Response {

        let path = request.url.path

        // Layer 1 exemption — pass through immediately
        if Self.exemptPaths.contains(path) {
            return try await next.respond(to: request)
        }

        let ua = request.headers.first(name: .userAgent)?.lowercased() ?? ""

        // Layer 4: Hard 403 — stop before any route handler runs
        if Self.aiBotPatterns.contains(where: { ua.contains($0) }) {
            return Response(
                status: .forbidden,
                headers: ["Content-Type": "text/plain"],
                body: .init(string: "Forbidden")
            )
        }

        // Pass legitimate request to the next handler
        let response = try await next.respond(to: request)

        // Layer 3: X-Robots-Tag — add to all served responses
        response.headers.add(name: "X-Robots-Tag", value: "noai, noimageai")

        return response
    }
}

Key points

  • AsyncMiddleware uses async throws — no EventLoopFuture or .map() chaining needed.
  • EXEMPT_PATHS check comes before the UA check. Without it, /robots.txt returns 403.
  • Return early for bot matches — do not call next.respond() after blocking.
  • response.headers.add() modifies the response in place. Call it after await next.respond().
  • request.headers.first(name: .userAgent) is the Vapor API for reading request headers by the built-in HTTPHeaders.Name enum.

Registration

Global — all routes

// Sources/App/configure.swift
public func configure(_ app: Application) async throws {
    // FileMiddleware first — serves Public/ before bot check
    app.middleware.use(FileMiddleware(publicDirectory: app.directory.publicDirectory))

    // Bot blocking applies to all routes not served by FileMiddleware
    app.middleware.use(AiBotMiddleware())

    try routes(app)
}

Route-grouped — selected routes only

Use app.grouped() to apply the middleware only to specific routes. Public routes on app directly are unaffected.

// Sources/App/routes.swift
func routes(_ app: Application) throws {
    // Public route — no bot blocking
    app.get { req async in "Hello, world!" }

    // Protected group — bot blocking applied
    let protected = app.grouped(AiBotMiddleware())
    protected.get("api", "data") { req async throws -> String in
        return "API response"
    }
    protected.get("premium") { req async throws -> View in
        try await req.view.render("premium")
    }
}

Package.swift

// Package.swift
// swift-tools-version: 5.9
import PackageDescription

let package = Package(
    name: "MyApp",
    platforms: [.macOS(.v13)],
    dependencies: [
        .package(url: "https://github.com/vapor/vapor.git", from: "4.0.0"),
        .package(url: "https://github.com/vapor/leaf.git",  from: "4.0.0"),
    ],
    targets: [
        .executableTarget(
            name: "App",
            dependencies: [
                .product(name: "Vapor", package: "vapor"),
                .product(name: "Leaf",  package: "leaf"),
            ],
            path: "Sources/App"
        ),
    ]
)

Leaf is only needed if you are using Leaf templates for the noai meta tag layer. If you are using a different templating engine or serving a Swift-rendered frontend, omit the Leaf dependency — the middleware works independently of the template engine.

Deployment

Vapor runs on Linux (Swift 5.9+) and macOS. Docker is the standard deployment path.

# Dockerfile
FROM swift:5.9-jammy AS build
WORKDIR /app
COPY . .
RUN swift build -c release

FROM swift:5.9-jammy-slim AS run
WORKDIR /app
COPY --from=build /app/.build/release/App .
COPY --from=build /app/Public ./Public
COPY --from=build /app/Resources ./Resources
EXPOSE 8080
CMD ["./App", "serve", "--env", "production", "--hostname", "0.0.0.0", "--port", "8080"]

Platforms: Fly.io (Dockerfile deploy, popular Vapor host), Railway (auto-detects Swift via Package.swift), Render (Docker deploy), Heroku (Swift buildpack), AWS EC2 / DigitalOcean (any Linux VPS with Swift installed). Nginx is optional in front of Vapor — if used, add AI bot blocking at the Nginx layer too (see the Nginx guide).

FAQ

Should I use AsyncMiddleware or Middleware for bot blocking?

Use AsyncMiddleware. Vapor 4 supports both the legacy EventLoopFuture-based Middleware protocol and the modern AsyncMiddleware protocol. AsyncMiddleware uses async throws, so you write respond(to:chainingTo:) as an async function with direct await calls — no .map() or .flatMap() chaining. Both work, but AsyncMiddleware is the recommended approach for all new Vapor 4 code.

Does Vapor middleware run before FileMiddleware serves robots.txt?

Middleware runs in the order it is registered. If AiBotMiddleware is registered before FileMiddleware, it intercepts /robots.txt before the file is served — the EXEMPT_PATHS check passes it through. If FileMiddleware is registered first, it serves /robots.txt directly and AiBotMiddleware never sees that request. Either ordering works as long as EXEMPT_PATHS is present.

How do I serve robots.txt without a Public/ directory?

Define a direct route: app.get("robots.txt") { req -> Response in ... } returning a Response with Content-Type: text/plain. Store the robots.txt content as a string constant in your source file. This works in Docker containers where Public/ may not be present at runtime.

How do I add noai meta tags in a Leaf template?

In your base Leaf template, add <meta name="robots" content="#(robots ?? \"noai, noimageai\")">. The ?? operator returns the fallback when robots is absent. In controllers, pass "robots": "index, follow" in the render context for pages that should not carry the noai directive. Omitting the key defaults to noai, noimageai.

Can I block bots only on API routes?

Yes. Use app.grouped(AiBotMiddleware()) and define protected routes on the grouped instance. Routes defined directly on app are not affected. This is the cleanest pattern for protecting API endpoints or authenticated content without blocking your public homepage or robots.txt.

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.