llms.txt: the practical setup guide
llms.txt is an emerging convention — in spirit next to
robots.txt — that gives AI agents a concise map of your
site: who you are, which pages matter, and optional deeper context via
llms-full.txt.
Why it matters
Agents that respect the file can skip noisy navigation and focus on authoritative product, policy, and docs URLs. It will not replace schema or content quality; it complements them as a clarity layer for machines.
Minimal setup
- Publish
https://yourdomain.com/llms.txtas plain text - Lead with a one-line product summary
- Link primary pages (home, about, docs, contact) with short descriptions
- Optionally add
llms-full.txtfor longer context - Keep URLs canonical and update the file when IA changes
Do
- Keep the file short and current
- Use absolute https URLs
- Separate human policies from API/agent discovery when needed
Don't
- Dump your entire sitemap into llms.txt
- Leave stale links after a redesign
- Expect llms.txt alone to win citations
Check it in Ad Cerebro
Agent Readiness checks surface missing or weak llms.txt
guidance. See Ad Cerebro’s own
llms.txt, then run a scan on your domain.