llms.txt explained: what it is, who reads it and whether you need one

llms.txt is a plain Markdown file at the root of a site, at /llms.txt. It gives a language model a short, curated map of the site: what it is, and which pages matter. Jeremy Howard of Answer.AI proposed it in September 2024, and it has spread quickly, especially across developer documentation.

What goes in it

The format is deliberately simple:

• An H1 with the name of the site or project.
• A blockquote with a one or two sentence summary.
• Optional paragraphs with anything a reader should know up front.
• H2 sections, each a list of links with a short note on what each page covers.

A section called "Optional" marks links a model can skip when it is short on context. That is the whole spec.

Who actually reads it

This is the honest part. None of the big AI assistants has said that its search crawler uses llms.txt to decide what to cite. Today it is mostly read by coding agents and developer tools that fetch docs on demand, and by people who paste it into a chat to give a model context.

So llms.txt is not a ranking trick. It will not get you cited if your robots.txt blocks AI search crawlers or your pages are empty without JavaScript.

Should you have one anyway

For most sites, yes, because it is cheap and it has no downside. Writing it forces you to name your most important pages and describe each in a sentence, which is also a good test of your site structure. And if assistants do start reading it, you are ready.

If you run developer docs, an API or a product people integrate with, the case is stronger. Coding agents already look for it, and a good llms.txt saves them from crawling your whole site to find the one page they need.

Making one in two minutes

Our llms.txt generator reads your sitemap and each page's title and description, then drafts the file for you. Nothing is invented: every line comes from your own pages. Edit the summary, drop the pages that do not matter, and upload it to your site root.