About
Jina AI's reader service (r.jina.ai) converts any URL to clean markdown for use in LLM pipelines. Often appears in logs when AI tools fetch pages via the Jina reader proxy. Also used by Jina's own search and embedding models.
Purpose
LLM-ready content extraction and AI embedding
User Agent String
Mozilla/5.0 (compatible; JinaBot/1.0; +https://jina.ai/reader)
How to Control in robots.txt
🚫 Block JinaBot
User-agent: JinaBot Disallow: /
✅ Allow JinaBot
User-agent: JinaBot Allow: /
Is JinaBot crawling your site?
Run a free scan to check if Jina AI's crawler is accessing your website.
Check if JinaBot is crawling YOUR site →