Tools / /llms.txt checker
/llms.txt checker
Check whether /llms.txt exists, uses useful sections, and gives agents a concise map of products, docs, policies, and actions.
Selected check · CONT-001-3 pts
What this checks
File exists
GET /llms.txt returns a readable text file.
Useful sections
Products, docs, actions, and policies are separated.
URL coverage
Links resolve on the same public origin.
Curation
Avoids dumping a full sitemap.
FAQ
- What should /llms.txt contain?
- It should be a concise public map of the pages agents need most: product overviews, docs, pricing, policies, support, integrations, and safe action paths.
- Should /llms.txt list every URL from sitemap.xml?
- No. A useful /llms.txt is curated. It should point agents to the most important public context instead of mirroring every crawlable URL on the site.
- Does /llms.txt replace sitemap.xml or robots.txt?
- No. Sitemap.xml helps discovery, robots.txt states crawler policy, and /llms.txt explains which public pages are most useful for agent reasoning.
- Does the checker read private pages linked from /llms.txt?
- The scanner is designed for public domains and safe public evidence. Do not place secrets, credentialed URLs, or non-public content in /llms.txt.