What Is llms.txt and Does Your Website Need One in 2026?

There is a new file making the rounds in AI marketing circles. It is called llms.txt. If you read the headlines, it sounds like the missing piece between your website and every AI chatbot that might cite your business. The truth is more specific and less dramatic.

Here is what the spec actually says and who uses it. Then you can decide if it belongs on your site.

TL;DR

llms.txt is a proposed markdown file at the root of your domain that tells large language models which pages carry the highest signal. The spec was proposed by Jeremy Howard in September 2024. As of mid-2026, major AI platforms including OpenAI, Google, and Anthropic do not fetch it at meaningful volume. Google's own AI optimization guide lists llms.txt among tactics you do not need. The file costs almost nothing to create and the exercise of writing one has real value, but it is not a citation shortcut and it does not replace structured data.

Key Takeaways

• llms.txt is a curated markdown index that points AI crawlers to your most important pages, not a robots.txt replacement that controls access.
• Major AI platforms including Google, OpenAI, and Anthropic do not fetch llms.txt at meaningful volume as of August 2026.
• Google's May 2026 AI optimization guide explicitly says llms.txt is not required for appearing in AI Overviews or AI Mode.
• The file is free to add and the process forces you to audit what content you would actually want an AI to cite.
• llms.txt does not replace structured data. Entity markup remains the primary language AI systems use to understand who you are.

What llms.txt actually is

llms.txt is a markdown file at the root of your domain. Jeremy Howard of Answer.AI proposed it in September 2024 because AI models have context windows too small to ingest full websites and raw HTML is full of noise from navigation, ads, and scripts.

The official proposal at llmstxt.org describes a markdown file giving AI systems a curated map of your content. An H1 with your project name. An optional blockquote. Then sections with linked markdown files, each with a one-line description. Think of it as a table of contents for a smart intern: here are the pages that matter, with a sentence on each.

What llms.txt is not

Robots.txt and llms.txt serve opposite purposes. Robots.txt controls what crawlers are allowed to access. llms.txt highlights what you want AI systems to notice. There is no blocking directive in the spec.

Search Engine Land described it as a choosing mechanism, not a restricting one. It is also not a substitute for structured data. Schema.org markup tells AI what your business is. llms.txt tells it which pages to read. We covered the markup layer in our guide on what structured data means for your business.

Who publishes llms.txt files and who reads them

Two different stories. On the publisher side, Anthropic, Stripe, Cursor, Cloudflare, Vercel, Supabase, and LangGraph all publish llms.txt files. Codersera's May 2026 guide estimates roughly 10 percent of websites have one.

On the reader side, server logs tell a different story. GPTBot fetches it rarely. ClaudeBot and Google-Extended effectively never request it. A 300,000-domain SERanking study found no effect on AI citation rates.

Google's May 2026 AI optimization guide lists llms.txt among tactics you do not need. The No Hacks AI user-agent landscape reference notes the irony: Anthropic, OpenAI, and Meta all publish llms.txt for their own docs but none read the format from other websites. The National Institute of Standards and Technology tracks AI standards and has not recognized llms.txt as a formal standard.

IDE coding agents like Cursor and Continue do read llms.txt for navigating software docs. It is a narrow but real use case.

The file family: one spec, four variants

The original proposal described one file. Implementers have added three companions since:

File	What It Contains	When to Use It
/llms.txt	Curated index with H1, summary, and links	Always. This is the map.
/llms-full.txt	Every linked page concatenated as one blob	For ingesting everything in one paste
/llms-ctx.txt	Pre-expanded context without URLs	FastHTML-style implementations
/llms-ctx-full.txt	Pre-expanded context with URLs preserved	Same as above, URL-aware

Most sites use the two-file pattern. llms.txt serves as the index and llms-full.txt as the deep read.

Should your business add one

The answer is yes, with the right expectations. The file costs almost nothing, maybe an hour of work. If a future AI platform starts reading llms.txt at scale, you are already indexed. If they never do, you spent that hour inventorying your best content for machine consumption. Either outcome is useful.

Here is what the process forces. List the pages carrying the signal you want AI to cite. Write a one-sentence description of each. Decide which content is secondary.

Most business owners, asked for that list, discover gaps they did not know they had. If those pages do not exist, the file will not fix the problem. It will reveal it.

llms.txt does not replace the signals AI systems use to decide who gets cited. To understand that process, see our breakdown of how ChatGPT picks businesses to recommend. Structured data, entity authority, and third-party mentions are the machinery. Our data on AI citation prediction shows which signals actually move the needle.

Sources cited in this analysis?

Frequently Asked Questions

Does Google read llms.txt files?

Google's May 2026 AI optimization guide says llms.txt is not required for AI Overviews or AI Mode. Googlebot occasionally fetches the file, but Google does not treat it as a ranking or citation signal. They recommend structured data and high-quality content instead.

Does adding an llms.txt file improve my AI citations?

No study has found a measurable connection between having an llms.txt file and higher citation rates. SERanking's 300,000-domain study found no effect. Major AI platforms do not fetch it at scale, so there is no mechanism for it to influence citations today.

What is the difference between llms.txt and robots.txt?

Robots.txt controls which crawlers can access your site and which paths they are permitted to visit. llms.txt highlights the specific pages you want AI systems to focus on. Robots.txt can block access to your content. llms.txt cannot. They serve opposite purposes.

Who proposed llms.txt and when?

Jeremy Howard, co-founder of Answer.AI and creator of FastAI, proposed the llms.txt standard in September 2024. The proposal is hosted at llmstxt.org and maintained as an open specification. Howard's background in machine learning documentation shaped the focus on clean, parseable markdown.

Do I need both an llms.txt and a sitemap.xml?

Yes. Sitemaps tell search engines about every indexable page on your site. llms.txt tells AI models which pages carry the highest signal. A sitemap is a complete inventory of what exists. An llms.txt is a curated selection of what actually matters.