What llms.txt actually is

It's a plain markdown file that sits at yoursite.com/llms.txt. Jeremy Howard, the developer behind fast.ai and Answer.AI, published the proposal on September 3, 2024 (llmstxt.org). The reasoning behind it: an AI model's context window is too small to read an entire website in one pass, so the file hands it a short, curated version instead, skipping the navigation bar, the cookie banner, and the JavaScript that makes up most of a real page.

The format itself is simple. An H1 with the site's name, a one line blockquote summary, and then H2 headed lists of links to the pages that actually matter, with a short note on each. For a B2B SaaS site, that's usually your docs, your pricing page, and the comparison pages your buyers actually read, not the full sitemap.

Adoption is climbing fast

People are genuinely doing this. Originality.ai tracked more than 3 million websites from June 2025 to May 2026 and counted llms.txt files going from 4,088 to 36,120 over that year, an 8.8x increase (Originality.ai, via PPC Land). A separate check of the Tranco top 1,000 domains found 8.7% already publish one, rising to 15.8% once you exclude domains that don't serve a public website at all (Rankability, July 2026).

A third, independently run study using HTTP Archive's crawl data found a flatter curve: 6.28% of the top 1,000 sites have one, dropping only to 5.07% by the top 1 million, "a 1.2 percentage point spread from the top 1k all the way to the top 1M" (Casey Burridge, June 2026). Three different methodologies, three different exact numbers, all landing in the same rough place: a real and growing minority of sites have one, size doesn't predict it much either way.

Almost nobody's file gets read

Here's the part the adoption numbers don't tell you. Ahrefs analyzed 137,210 domains on its Web Analytics platform, found 28% of them publish an llms.txt file, and then checked server logs to see what actually requested those files in May 2026. The answer: 97% of them received zero requests. Nothing fetched them at all (Ahrefs, June 2026).

Worth noticing

Of the 3% of files that did get requested, 96% of those requests came from bots, not people, and only 19.5% of that bot traffic came from named AI tools. Another 12% was the industry studying itself, GEO tools, llms.txt checkers, researchers. Ahrefs' own detail sums it up best: Slackbot alone fetched llms.txt files more often than PerplexityBot did.

None of this means llms.txt is a scam or a waste of five minutes. It means the file exists mostly as a convention people are adopting ahead of confirmed demand, not in response to it.

Where Google and OpenAI actually stand

Google has said this on the record. On the Search Off the Record podcast, John Mueller explained that self-reported files like llms.txt can't help a model choose one site over another: "By design, they can't trust what is here as a way of differentiating between different websites" (Search Engine Journal). Google Search ignores it entirely, for rankings and for AI Overviews. Mueller pointed back to the same two things that have always mattered: normal HTML pages and internal links.

OpenAI hasn't made a comparable public statement, but its own crawler documentation is the tell. GPTBot, OAI-SearchBot, and ChatGPT-User are all controlled through robots.txt. llms.txt is never mentioned. As of mid 2026, no major AI provider, not OpenAI, not Anthropic, not Google, not Perplexity, has confirmed reading the file at inference time.

Should you bother anyway

Probably, but for a smaller reason than most people selling llms.txt generators will tell you. It costs close to nothing: a static file, five to ten minutes of writing, zero engineering risk. It doesn't hurt anything either way. The honest case for adding one isn't "this will get you cited," it's "this is a five minute task with no downside, and the convention might matter more in a year or two than it does right now."

What actually moves whether an AI system can reach and use your site at all is the boring, checkable stuff underneath: whether GPTBot and ClaudeBot are blocked in robots.txt, whether your homepage renders real text without JavaScript, whether you have basic schema markup. I built a free checker that runs through exactly that, llms.txt included, since it's one data point among several, not the whole story.

A minimal template you can copy

If you want to add one anyway, here's a starting structure. Swap in your own product name, one line summary, and the pages that actually matter to a buyer.

# Acme
> Acme is project management software for construction teams. This file lists the pages most relevant to understanding what Acme does and how it's priced.

## Docs
- [Getting started](https://acme.com/docs/getting-started): setup and first project walkthrough
- [API reference](https://acme.com/docs/api): full endpoint documentation

## Product
- [Pricing](https://acme.com/pricing): plans and per seat cost
- [Acme vs Procore](https://acme.com/compare/procore): feature and pricing comparison

## Optional
- [Blog](https://acme.com/blog): product updates and case studies

Keep it short, keep it accurate, and don't treat writing it as the finish line. It's the smallest item on a much longer list.

FAQ
What is llms.txt?
A plain markdown file at yoursite.com/llms.txt, proposed by Jeremy Howard in September 2024, meant to give AI systems a short, curated summary of a site instead of forcing them to process full HTML pages.
Does ChatGPT or Google actually use llms.txt?
No major AI provider has confirmed using it as of mid 2026. Google's John Mueller said Search ignores it entirely, and OpenAI's official crawler documentation for GPTBot, OAI-SearchBot, and ChatGPT-User never mentions llms.txt at all, only robots.txt.
Is it worth adding llms.txt to my site anyway?
It costs almost nothing to add and does not hurt anything, so there is no real downside. Just do not expect it to move your AI visibility on its own. An Ahrefs analysis of 137,210 domains found 97% of published llms.txt files received zero requests in May 2026.

Primary sources: llmstxt.org (Jeremy Howard, September 2024) · Originality.ai adoption tracking via PPC Land (July 2026) · Rankability Tranco top 1,000 analysis (July 2026) · Casey Burridge, HTTP Archive adoption study (June 2026) · Ahrefs, 137,210 domain server log analysis (June 2026) · Search Engine Journal on John Mueller, Search Off the Record. All sources accessed August 1, 2026.