LLMS.txt Generator

An llms.txt file is a markdown file that tells AI agents what a site contains and where its useful pages are. It sits at /llms.txt, it is written to be read by both people and machines, and it is a proposal rather than a ratified standard. No search engine or assistant is obliged to read one.

The problem it addresses is narrow and real. A web page wraps its content in navigation, markup and scripts, so an agent working inside a limited context window spends tokens stripping all of that away before it reaches anything worth having. An llms.txt file hands it a short curated map in plain markdown instead. The file itself stays small; the detail lives behind the links and is fetched only when something needs it.

Jeremy Howard published the proposal in September 2024 and revised it to v2 in August 2026. The generator above writes the v2 shape.

The spec names one required element and fixes the order of everything else:

1. An H1 with the name of the site or project. This is the only part the spec actually requires.
2. A blockquote summarising the site, carrying whatever an agent needs in order to make sense of the rest of the file.
3. Any amount of ordinary markdown giving more detail, as paragraphs or lists, but no headings.
4. Any number of H2 sections, each holding a markdown list of links, and each link optionally followed by a colon and a note on what the page covers.

# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url)

The notes after each link are the part worth spending time on. They are what an agent reads to decide a page is relevant before it spends anything fetching it.

A section headed Optional is a convention for links that can be skipped when a shorter context is needed. Under v1 that heading carried mechanical meaning for context-expansion tooling. v2 removed the tooling from the proposal and the special meaning went with it, so the heading is now a hint to the reader and nothing more.

The root, at yourdomain.com/llms.txt, is the normal answer. v2 also allows a file on any subpath, so /docs/llms.txt covers the pages under /docs/, and where more than one file applies the most specific one wins. That is what lets a project controlling only a path, a GitHub Pages site for instance, publish one at all.

v2 also answers the question v1 left open, which is how an agent finds the file without guessing. Two standard link relations do it: rel="describedby" points at the llms.txt covering a page, and rel="alternate" type="text/markdown" points at that page's markdown version. Either can be an HTML <link> element or an HTTP Link: response header. The header form also covers non-HTML files and can be set in your CDN without touching a single page.

The markdown versions are the other half of the v2 proposal: serve a clean markdown copy of a page at the same URL, either with .md appended or with the extension replaced. v1 specified only the first form. Publishing tools had already diverged, so v2 accepts both.

This is where most writing on llms.txt overreaches, so it is worth separating what can be checked from what cannot.

What is verifiable:

• The AI labs publish llms.txt files for their own developer documentation, including OpenAI, Anthropic and Google.
• Chrome ships an llms.txt audit in Lighthouse, under its agentic browsing checks. Read the detail carefully: it flags a server error on the file, and marks the audit Not Applicable when the file is simply absent, because Google describes publishing one as optional and calls the file an emerging convention. A missing llms.txt is not scored against you.
• Documentation platforms now generate the file automatically, and coding agents fetching library documentation are the heaviest users of it.

What is not established:

• No assistant vendor publishes a commitment to read a third-party llms.txt when it composes an answer. Publishing a file for your own docs is not the same as consuming everybody else's, and the two get conflated constantly. Treat any claim that a named assistant reads llms.txt as unconfirmed unless that vendor says so directly.
• Circulating figures for adoption rates and citation uplift come from third-party samples rather than from any platform, so they are contested. We do not repeat them.

The practical read is that llms.txt is cheap hygiene with a clear use for documentation and agents, and an unproven effect on whether an assistant names you in an answer. Our fuller assessment is in the guide to llms.txt and AI visibility.