LLMs.txt in 2026: The Full Guide

Yahel Oren
Data Scientist

Table of Contents

• What llms.txt actually is
• The bad: what every honest study has found
• The good: llms.txt is a B2A protocol, not a GEO protocol
• Where B2A surfaces matter today
• How to build llms.txt correctly
• Best practices and common mistakes
• The bottom line
• FAQs

LLMs.txt in 2026: The Full Guide

Bots Aren't Fetching It Yet. Here's Why It's Still a B2A Must-Have.

We analyzed over 500M LLM bot traffic events across the brands we monitor at Limy. One finding stands out: AI search crawlers are almost never fetching /llms.txt. GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, and Google-Extended overwhelmingly skip the file and crawl HTML directly. That's the empirical reality in May 2026, and it doesn't match the noise in the GEO space.

It also doesn't change our recommendation. We tell every brand we work with to ship one.

The reason is simple: llms.txt is not the SEO play it's being sold as. It's a Business-to-Agent (B2A) play - the first standardized way for a brand to publish a machine-readable surface that AI agents can route on.

What llms.txt actually is

llms.txt is a proposed standard, introduced by Jeremy Howard of Answer.AI in September 2024, for a Markdown file at the root of your domain - https://example.com/llms.txt - that gives AI systems a curated index of your most important content with one-line descriptions.

There's also llms-full.txt, a longer companion that embeds the actual content of those linked pages so an agent can ingest everything in a single fetch.

The bad: what every honest study has found

If you're shipping llms.txt to improve AI search visibility - to get cited more in ChatGPT, Perplexity, Claude, Gemini, or Google AI Mode - the evidence is brutal.

• Adoption is low and not growing fast. A SE Ranking study of 300,000 domains found a 10.13% adoption rate. After eighteen months of industry conversation, llms.txt is on roughly one in ten sites.
• Crawler interest is negligible from the search/answer bots. We monitored over 500M AI bot visits across a 90-day window - only 408 targeted llms.txt directly. That's neglectible for all AI crawler traffic.
• Google has said no, on the record. In July 2025, Google's Gary Illyes confirmed Google doesn't support llms.txt and isn't planning to, and John Mueller compared it to the discredited keywords meta tag.

The good: llms.txt is a B2A protocol, not a GEO protocol

The same file that does almost nothing for ChatGPT search citations is doing real work in a different layer of the AI stack: the agentic web - the layer where AI agents act on behalf of users, fetch context, choose tools, and complete tasks.

IDE agents fetch llms.txt routinely. Cursor, Windsurf, Claude Code, GitHub Copilot, Cline, Aider - they all look for /llms.txt and /llms-full.txt when pointed at a documentation site.

Where B2A surfaces matter today

llms.txt is most often discussed in the context of developer tools, because that's where the agentic-web stack matured first. But the B2A pattern generalizes well past dev tools, and we're already advising clients across categories on how to think about it.

1. SaaS and developer platforms

The mature case.

2. E-commerce and agentic commerce

The fastest-growing use case.

3. Financial services and insurance

4. Pharma and healthcare

5. Travel and hospitality

6. B2B and enterprise software

7. News, research, and publishing

How to build llms.txt correctly

The format

llms.txt is a Markdown file with a specific structure. The Answer.AI spec is loose, but the convention that's emerged across the dev-tool ecosystem looks like this:

# Brand or product name

> One-sentence summary of what this brand/product is and who it's for.

## Section name

- [Page title](https://example.com/page): One-line description of what's on this page and when an agent should fetch it.

The build process, step by step

1. Identify the canonical pages.
2. Group into 4–7 sections.
3. Write the H1 and summary.
4. Write descriptions.
5. Decide on llms-full.txt.

Best practices and common mistakes

• Keep it curated.
• Write descriptions for context, not SEO.
• Don't create indexable Markdown duplicates.
• Update quarterly.
• Don't gate it.

The bottom line

llms.txt in May 2026 is not a GEO ranking factor. Brands shipping llms.txt hoping for a citation boost in ChatGPT or Perplexity will be disappointed.

FAQs

Is llms.txt a Google ranking factor?
No. Google has publicly stated it doesn't support llms.txt and isn't planning to.

Does llms.txt improve my citation rate in ChatGPT or Perplexity?
Crawler interest is negligible from the search/answer bots.

So why ship it at all?
Because it's the first widely-adopted B2A standard, and the agentic web is where AI traffic is heading.