Skip to content
Guides/PocketBase

How to Block AI Bots on PocketBase: Complete 2026 Guide

PocketBase is a single Go binary that ships a full backend — SQLite database, REST API, real-time subscriptions, file storage, and an admin panel. Bot blocking uses PocketBase's hook system: app.OnBeforeServe() gives you access to the underlying Echo router before the server starts accepting requests — you add a standard Echo middleware function there. The pb_public/ directory handles robots.txt with no code at all.

Two deployment modes

Source mode — you write Go code extending PocketBase. Full access to hooks and Echo middleware. All 4 layers available.

Binary mode — running the pre-built./pocketbase binary with no Go source. Only Layer 1 (robots.txt in pb_public/) is available. Use nginx/Caddy for Layers 3–4.

Four protection layers

1
robots.txt — pb_public/robots.txt — served automatically at /robots.txt, zero config
2
noai meta tag — SPA index.html (pb_public/) or Go html/template in custom HTML routes
3
X-Robots-Tag header — OnBeforeServe() → Echo middleware → c.Response().Header().Set()
4
Hard 403 block — OnBeforeServe() → Echo middleware → c.String(403, "Forbidden") + return

Layer 1: robots.txt

PocketBase automatically serves everything in pb_public/ at the web root — no route definition needed. Place robots.txt there and it is immediately accessible at /robots.txt.

# pb_public/robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: Applebot-Extended
User-agent: PerplexityBot
User-agent: Diffbot
User-agent: cohere-ai
User-agent: FacebookBot
User-agent: omgili
User-agent: omgilibot
User-agent: Amazonbot
User-agent: DeepSeekBot
User-agent: MistralBot
User-agent: xAI-Bot
User-agent: AI2Bot
Disallow: /

In binary mode this is the only available protection layer on the PocketBase process itself. For hard blocking in binary mode, configure nginx or Caddy in front of PocketBase and apply bot blocking at the reverse proxy layer.

Layer 2: noai meta tag

PocketBase does not render HTML pages by default. The meta tag approach depends on how your frontend is deployed.

SPA in pb_public/ (most common)

If your React, Vue, or Svelte app is built into pb_public/, add the noai meta tag to index.html in your frontend source before building:

<!-- index.html (Vite/CRA/SvelteKit frontend source) -->
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />

  <!-- AI bot training opt-out -->
  <meta name="robots" content="noai, noimageai" />

  <title>My App</title>
</head>

Custom Go HTML routes (source mode)

Register custom routes via OnBeforeServe() to serve server-rendered HTML with per-page robots control:

// main.go — custom HTML route with robots meta
import (
    "html/template"
    "strings"
    "github.com/pocketbase/pocketbase"
    "github.com/pocketbase/pocketbase/core"
)

var pageTmpl = template.Must(template.New("page").Parse(`
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="robots" content="{{.Robots}}">
    <title>{{.Title}}</title>
</head>
<body>{{.Body}}</body>
</html>`))

app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
    e.Router.GET("/about", func(c echo.Context) error {
        var buf strings.Builder
        pageTmpl.Execute(&buf, map[string]string{
            "Title":  "About",
            "Robots": "index, follow", // override for this page
            "Body":   "<h1>About</h1>",
        })
        return c.HTML(200, buf.String())
    })
    return nil
})

Layers 3 & 4: OnBeforeServe middleware

PocketBase exposes the underlying Echo router via the OnBeforeServe() hook. Register a global Echo middleware there — it intercepts all requests before any PocketBase route handler runs.

// main.go
package main

import (
    "log"
    "net/http"
    "strings"

    "github.com/labstack/echo/v5"
    "github.com/pocketbase/pocketbase"
    "github.com/pocketbase/pocketbase/core"
)

var aiBotPatterns = []string{
    "gptbot", "chatgpt-user", "oai-searchbot",
    "claudebot", "anthropic-ai", "claude-web",
    "google-extended", "ccbot", "bytespider",
    "applebot-extended", "perplexitybot", "diffbot",
    "cohere-ai", "facebookbot", "meta-externalagent",
    "omgili", "omgilibot", "amazonbot",
    "deepseekbot", "mistralbot", "xai-bot", "ai2-bot",
}

// Always allow — crawlers must be able to read robots.txt
var exemptPaths = map[string]bool{
    "/robots.txt": true,
    "/sitemap.xml": true,
    "/favicon.ico": true,
}

func aiBotMiddleware(next echo.HandlerFunc) echo.HandlerFunc {
    return func(c echo.Context) error {
        path := c.Request().URL.Path

        // Layer 1 exemption — pass through immediately
        if exemptPaths[path] {
            return next(c)
        }

        ua := strings.ToLower(c.Request().Header.Get("User-Agent"))

        // Layer 4: hard 403 block — do not call next()
        for _, pattern := range aiBotPatterns {
            if strings.Contains(ua, pattern) {
                return c.String(http.StatusForbidden, "Forbidden")
            }
        }

        // Call handler for legitimate requests
        if err := next(c); err != nil {
            return err
        }

        // Layer 3: X-Robots-Tag on every legitimate response
        c.Response().Header().Set("X-Robots-Tag", "noai, noimageai")
        return nil
    }
}

func main() {
    app := pocketbase.New()

    app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
        // Register global middleware — applies to ALL routes
        e.Router.Use(aiBotMiddleware)
        return nil
    })

    if err := app.Start(); err != nil {
        log.Fatal(err)
    }
}

Key points

  • app.OnBeforeServe() fires once when the HTTP server is ready but before it starts accepting connections. It is the correct hook for registering middleware — not OnAfterBootstrap() (too early, no router yet).
  • e.Router.Use() applies the middleware globally to all routes — PocketBase API (/api/), admin panel (/_/), and any custom routes.
  • c.Response().Header().Set() must be called after next(c) — before it, the response headers have not been written yet but setting them early may be overwritten by the handler.
  • For binary mode deployments, skip this Go code — use nginx or Caddy in front of PocketBase and apply bot blocking at the reverse proxy layer instead.
  • PocketBase v0.22+ uses Echo v5. For older PocketBase versions using Echo v4, the middleware signature is identical — echo.MiddlewareFunc takes the same func(next HandlerFunc) HandlerFunc shape.

Selective route blocking

To block bots only on custom routes (not the PocketBase API or admin panel), apply the middleware to a specific route group instead of globally:

app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
    // Custom app routes — bot blocking applied
    appGroup := e.Router.Group("/app", aiBotMiddleware)
    appGroup.GET("/dashboard", dashboardHandler)
    appGroup.GET("/reports", reportsHandler)

    // PocketBase /api/ and /_/ are NOT in this group — unaffected
    return nil
})

go.mod

module myapp

go 1.22

require (
    github.com/pocketbase/pocketbase v0.22.0
    github.com/labstack/echo/v5      v5.0.0-20230722203903-ec5b858dab61
)

Echo v5 is a transitive dependency of PocketBase — you do not need to add it separately if you are only importing github.com/labstack/echo/v5 for the middleware type. Run go mod tidy after editing go.mod.

Verification

# Layer 1 — static robots.txt from pb_public/
curl https://yoursite.com/robots.txt

# Layer 3 — X-Robots-Tag on a real page
curl -I https://yoursite.com/
# Expected: X-Robots-Tag: noai, noimageai

# Layer 4 — hard 403 on bot user-agent
curl -A "Mozilla/5.0 (compatible; GPTBot/1.0)" -I https://yoursite.com/
# Expected: HTTP/1.1 403 Forbidden

# robots.txt must be exempt
curl -A "GPTBot" -I https://yoursite.com/robots.txt
# Expected: HTTP/1.1 200 OK

FAQ

Where does PocketBase serve static files like robots.txt?

PocketBase automatically serves everything in pb_public/ at the web root — no route definition needed. Place robots.txt at pb_public/robots.txt and PocketBase serves it at /robots.txt with the correct Content-Type: text/plain header. This is the same directory where you deploy a React or Vue SPA build. The path is relative to the directory where the pocketbase binary runs, not the Go source directory.

How do I add global middleware to PocketBase?

Use app.OnBeforeServe().Add(func(e *core.ServeEvent) error { ... }) and call e.Router.Use(yourMiddleware) inside the hook body. OnBeforeServe fires after PocketBase has registered all its internal routes, so your middleware applies to all incoming requests in the correct order. Do not use OnAfterBootstrap — the router is not ready at that point.

Will this middleware affect the PocketBase admin panel and API?

Yes — e.Router.Use() applies globally to all routes including /_/ (admin panel) and /api/ (PocketBase REST API). AI bots without admin credentials would be blocked before reaching authentication anyway. If you need the PocketBase health check at /api/health accessible to all crawlers, add it to EXEMPT_PATHS. For selective protection, use e.Router.Group("/prefix", middleware) instead of global Use().

How do I add noai meta tags to pages served by PocketBase?

Two approaches: (1) SPA in pb_public/ — edit index.html in your frontend source to include the noai meta tag before running your build command (npm run build). The static HTML file carries the tag. (2) Custom Go routes — register routes via OnBeforeServe() and use Go's html/template package to render HTML strings with a robots variable, then return c.HTML(200, rendered). Pass robots: 'noai, noimageai' as the default and 'index, follow' for specific pages.

Does this work for the pre-built PocketBase binary?

The robots.txt approach (Layer 1) works with any deployment — just place the file in pb_public/. The Go middleware (Layers 3–4) requires you to compile PocketBase from source with your extension code. If you are running the pre-built ./pocketbase binary, add nginx or Caddy as a reverse proxy in front of it and implement bot blocking at the proxy layer — see the Nginx or Caddy guides for the configuration.

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.