A second, heavier file that holds the actual text of your selected pages as Markdown, so an AI system can read the content itself, not just the links. Pages are fetched one by one: a page that fails, refuses the fetch or is blocked by its own robots.txt is listed here and left out — never invented. The first 25 selected pages are included.
What llms.txt is. A community proposal: a short Markdown file that lists your best pages for AI systems. It is not an official standard, and Google says its AI features need no special files — so treat it as a low-cost extra, not a ranking trick, and expect no promised visibility from it.
Curate, do not copy. The file works best as a short list of your best pages — this tool warns past about 50. Group pages into named sections, push the secondary ones into the Optional section, and write a one-line description that says what is actually on each page.
Check before you publish. A sitemap only lists addresses; it does not promise they still work. The checker follows each page and flags errors, redirects, noindex pages and robots.txt blocks, because listing a page an AI crawler is told not to touch helps nobody. Review the finished file, keep your sitemap.xml and robots.txt in place, and upload llms.txt to your site root.
Turn your sitemap into a curated llms.txt: check pages for errors and noindex, keep only your best pages, and download the file. Free, no sign-up.
100% free · No sign-up · No daily limits · No watermarks · Processed in your browser — files never leave your device
Load your sitemap, keep only your best pages, check them for noindex, robots blocks, redirects and errors, group them into sections, and download a clean llms.txt — plus an optional llms-full.txt with your pages’ real text as Markdown. Free, in your browser.
A tools site with 290 sitemap URLs loads the whole index, unticks the thin and duplicate pages (language versions are flagged automatically), groups what is left into Tools, Guides and Company sections, runs the check to catch a noindexed utility page and a redirected old address, and ends with about 40 links in a file under 4 KB.
Everything runs in your browser. Pages are fetched directly first — which also lets the checker read response headers such as X-Robots-Tag — and when a site blocks that, the tool falls back to this site’s protected fetcher for the page itself. Sitemaps are parsed as real XML, including indexes and gzipped files. The output follows the llms.txt format: an H1 with the site name, an optional blockquote summary and notes, then one Markdown link list per section, with Optional always last and parentheses in URLs percent-encoded so links never break.
A plain-text file at your site’s root (yoursite.com/llms.txt) that lists your most important pages with short descriptions, written for AI tools and language models rather than search crawlers. It was proposed in 2024 and support is still patchy — treat it as a tidy, cheap addition, not a ranking lever.
Fewer than you think. The format works best as a short list of your best pages — a few dozen at most. The tool warns past about 50 selected pages; put secondary pages in the Optional section instead of deleting them.
No. Google has said llms.txt is not used for Search or AI Overviews. Some AI assistants and developer tools do read it, which is why a clean file is still worth the few minutes it takes.
The same pages, but with their actual main content converted to Markdown, so a model can read the substance without crawling. It is much larger than llms.txt and entirely optional — build it only if you want it, and only for pages your robots.txt allows.
Yes — the Import tab reads an existing file from an address or pasted text and rebuilds its sections, titles and descriptions as an editable list, so you can refresh it instead of starting over.
What is missing, broken, or could be better on this tool? Tell us — we read every suggestion.
✅ Thanks — your suggestion has been received. Suggestions like yours shape what we build next.