An llms.txt file is a Markdown file at the root of a website that gives language models a short, curated list of the pages worth reading. This llms.txt generator builds a first draft from your homepage and sitemap, and the llms.txt validator checks a published file against the llmstxt.org proposal. Before you spend time on it, read the section on what the file does and does not do: the evidence that AI systems read it is thin.
What does an llms.txt file look like?
It is plain Markdown with a fixed order: one H1 with the site or project name, a blockquote with a one-sentence summary, optional paragraphs, then H2 sections that each hold a list of links in the form - [Title](URL): note. A section named "Optional" marks links a model can skip when it is short on context. A minimal llms.txt example:
# Example Co
> Example Co makes accounting software for freelancers in the UK and Ireland.
## Product
- [Features](https://example.com/features): Invoicing, expenses and tax estimates
- [Pricing](https://example.com/pricing): Plans and what each includes
## Docs
- [Getting started](https://example.com/docs/start): Set up an account and import bank data
## Optional
- [Blog](https://example.com/blog): Product news and guidesHow the generator builds your draft
- Reads your homepage for the site name (from WebSite structured data, og:site_name or the title) and the summary (from the meta description).
- Reads robots.txt to find your sitemap, then falls back to common sitemap paths and, failing that, to links on the homepage.
- Picks up to 30 pages in sitemap order, favouring top-level pages and spreading the rest across sections. On sites with language folders it keeps one language.
- Fetches each page and takes its title and meta description. Pages that return errors, redirect elsewhere or carry noindex are left out.
- Groups pages by their first folder, and moves blog posts, news, legal pages and similar secondary content into "Optional".
The result is a starting point. Our view: the value of llms.txt is in the curation, so reorder the links so the most useful come first, rewrite notes that are vague, and remove pages a model would not need.
What the validator checks
| Check | Passes when |
|---|---|
| File exists | /llms.txt returns HTTP 200 |
| Content type and no HTML | Served as text/plain or text/markdown, with no HTML (a common sign of a single-page app returning its shell) |
| Single H1 first | The first line is # Name and there is no other H1 |
| Blockquote summary | A > line follows the H1 directly |
| Link lists | H2 sections contain Markdown link list items |
| Size | Under 100 KB |
| Links resolve | A sample of up to 10 links on your own site return HTTP 200 |
Does llms.txt help SEO or AI visibility?
There is no evidence that it does today. Google says its AI features need no special files or markup (AI features and your website), and Google representatives have said Search does not use llms.txt. No major AI provider has confirmed that its crawlers read the file. Research shows the same picture from the other side: an Ahrefs study in 2026 found that about 97% of llms.txt files received no requests from AI bots at all.
That does not make it harmful. It costs little, some coding assistants and agent tools look for it, and it doubles as an internal map of what you consider important. If AI visibility is the goal, spend the time first on what crawlers demonstrably use: allow OAI-SearchBot and the other search crawlers in robots.txt, serve content in the HTML rather than only through JavaScript, and answer questions plainly. The AI crawlers guide explains which bot does what, and the llms.txt guide covers the format and the debate in more depth.
Limits
The generator reads at most 30 pages and stops after about 45 seconds, so large sites get a sample rather than a complete index. It does not run JavaScript, so titles set in the browser are not seen. Section names come from URL folders and may need renaming. The validator follows llmstxt.org, which is a proposal rather than a ratified standard, so a warning here means "differs from the proposal", not "broken for every reader".
Frequently asked questions
- Where does the llms.txt file go?
- At the root of your site, so it loads at https://your-domain/llms.txt. Serve it as plain text or Markdown in UTF-8. A file in a subfolder can describe that section only, the same way a docs site might publish one under /docs/.
- What is the difference between llms.txt and llms-full.txt?
- llms.txt is a short index: a title, a one-line summary and lists of links. llms-full.txt is a convention some documentation sites use to put the full text of their pages into one large Markdown file. This tool generates and validates llms.txt.
- Does Google use llms.txt?
- No. Google has said Search does not use llms.txt, and its documentation says no special files are needed to appear in AI Overviews or AI Mode. Those features rely on normal crawling and indexing.
- Can llms.txt block AI crawlers or opt out of training?
- No. It is a reading list, not an access rule. To allow or block crawlers such as GPTBot, OAI-SearchBot or ClaudeBot, use robots.txt.
- How often should I update it?
- Whenever your key pages change: a new product, a renamed docs section, retired URLs. Regenerate a draft, keep your hand edits, and run the validator so broken links do not creep in.