While robots.txt says what crawlers may not do, the emerging /agents.txt convention says what automated clients can do — which endpoints are open, how to buy, where to get machine-readable docs. It's the welcome mat of the agentic web.
A minimal template
# agents.txt — automated client guide for https://your-site.com Contact: hello@your-site.com Docs-Machine-Readable: /llms.txt /llms-full.txt Catalog: /products.json Payments: x402 (/.well-known/x402), Stripe Checkout links in catalog Crawl-Policy: see /robots.txt Updated: 2026-08-26
Why it matters even if few parsers read it yet
- Signal of intent: agent operators triage sites by their machine-facing files. A present, maintained agents.txt reads as "this site wants agent business."
- Consistency signal: sites that keep agents.txt, llms.txt, and robots.txt aligned score higher on every readiness rubric we know.
- Future-proofing: conventions here converge fast; having the file means adopting the winning format is a one-line change.
Keep it honest
Only claim what's true: if you list Docs-Machine-Readable paths, they must exist. Our audits check every declared surface — a dead link in agents.txt costs you credibility with exactly the audience that automates everything.
Does your site pass the full surface check?
Free audit — all six surfaces gradedWeekly monitoring: €19/mo