Make Your Website Visible to ChatGPT & Perplexity

A growing share of buyers never open Google anymore — they ask an assistant. If your site can't be read by machines, you don't rank badly in that channel: you don't exist in it. The good news: the technical bar is low. Here is the complete checklist we run in our audits.

The six surfaces that matter

Surface	What it does	Skip it and…
/robots.txt	Tells AI crawlers (GPTBot, ClaudeBot, PerplexityBot) what they may read.	Risk being ignored or blocked by default.
/llms.txt	A machine-readable summary of your site: products, prices, buy-links.	Assistants guess wrong about what you sell.
/llms-full.txt	The detailed version of llms.txt for deep retrieval.	Fine-grained questions get shallow answers.
/sitemap.xml	The canonical list of your pages.	New pages surface slowly or never.
/.well-known/x402	Machine-readable payment config so agents can pay you.	Agents can browse but not buy.
/agents.txt	Declares which automated clients are welcome and how.	Agent operators treat you as unfriendly.

Fix #1: robots.txt — let the right crawlers in

Many sites block AI crawlers by accident (or by an overzealous CDN rule). The minimum viable policy:

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

Sitemap: https://your-site.com/sitemap.xml

If you deliberately want to keep training crawlers out but allow assistants in, split the rules per user-agent instead of blocking everything — visibility requires read access.

Fix #2: llms.txt — answer before it's asked

An assistant with limited context will often read only one file about you. Make it count: list who you are, what you sell, at which prices, and where to buy — in plain Markdown:

# Your Company

You help small businesses automate X.

## Products
- **Widget Pro** — €29 — buys here: https://your-site.com/buy-widget-pro
- **Consulting** — from €150/h — details: https://your-site.com/consulting

## How to buy
Open any Stripe link above. Card payment, instant delivery by email.

That last line matters: tell the agent's buyer how completion works, not just what you offer.

Fix #3: payment readiness (x402)

AI agents increasingly transact programmatically via the x402 protocol (HTTP 402 + USDC on Base). A minimal /.well-known/x402 manifest advertises what you accept:

{
  "accepts": [{
    "scheme": "gateway",
    "network": "8453",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "maxAmountRequired": "0.01",
    "payTo": "0xyour-wallet"
  }]
}

Card-only storefronts should still expose this file pointing to their checkout URLs — discoverability is the point, even if the final payment is human-completed.

How fast do results show up?

Crawler permissions take effect within days; llms.txt content is typically picked up on the next fetch cycle. Expect measurable movement within 2–4 weeks after fixing all six surfaces.