An llms.txt file is a plain-text markdown file you place at your site's root (yoursite.com/llms.txt) that gives AI models a short, curated map of your most important content: what the site is, and which pages are worth reading first. It works like robots.txt or sitemap.xml, except the audience is a language model instead of a search-engine crawler.
That is the whole idea in one sentence. The harder question, and the one most articles on this topic dodge, is whether it actually changes anything. Below is the exact format, a template you can copy today, a step-by-step setup, and a straight answer on whether it is worth your time.
What llms.txt Actually Is
The proposal comes from Jeremy Howard (Answer.AI) and was published at llmstxt.org in September 2024. The pitch: language models have limited context windows and no reliable way to tell your marketing copy from your API reference. An llms.txt file hands the model a short, structured summary instead of making it guess by crawling your whole site.
It is not a technical standard enforced by any browser or protocol. It is a convention, the same way robots.txt started as a convention before search engines agreed to honor it. Nothing forces an AI crawler to read your llms.txt file, and that distinction matters for everything that follows.
The Exact Format (Copy This Structure)
The spec is deliberately narrow. A valid llms.txt file follows this order:
- An H1 with your site or project name. This is the only required section.
- A blockquote with a one- or two-line summary. What the site is, in plain language.
- Optional context paragraphs. A few sentences of background, no headings.
- H2-delimited sections, each a list of markdown links.
[Page name](url): short description of what it covers. - An "Optional" H2 section (if needed). Signals to the model that this content can be skipped in a shorter context window.
Here is a minimal, working example (the two-hash lines are H2 headings in the real file):
# Acme Analytics
> Acme Analytics is a self-serve product analytics tool for SaaS teams. This file
> points AI models to our most important documentation and product pages.
Acme Analytics helps teams track user behavior, run funnels, and set up
retention dashboards without writing SQL.
[H2] Docs
- [Quickstart](https://acme.com/docs/quickstart): Get a dashboard running in 10 minutes.
- [API Reference](https://acme.com/docs/api): Full endpoint documentation.
- [Pricing](https://acme.com/pricing): Current plans and limits.
[H2] Optional
- [Changelog](https://acme.com/changelog): Release history, safe to skip for short context.
Some sites also publish llms-full.txt, a longer companion file with the complete text of key pages inlined rather than just linked. That is not part of the original spec, but it has become a common extension for documentation-heavy sites (Anthropic and Cloudflare both do a version of this).
Does llms.txt Actually Work? The Honest Answer
Here is where most explainers go quiet, because the data is not flattering. As of mid-2026, none of the major AI providers, OpenAI, Anthropic, Google, or Meta, have committed to reading or prioritizing llms.txt files. Google has said directly that these files are not necessary for AI visibility in Search, even as its own Chrome Lighthouse tool added an experimental check for one. Independent crawls of domains that host an llms.txt file have found that the overwhelming majority of those files are never requested by any bot at all.
<mark class="km-highlight" style="--hl:#FEF08A;background:#FEF08A">llms.txt is not currently a ranking or citation lever any major AI system reads.</mark> Treat that as the baseline expectation, not a discouragement from ever using one.
So why bother? Three reasons that hold up even with that caveat:
- It costs almost nothing to create and maintain. A well-structured
llms.txtfile is a few minutes of work once your content map is organized. There is no meaningful downside to publishing one. - Some AI-native tools already parse it. Documentation platforms (Mintlify, GitBook) and a growing set of dev-tool crawlers do read it, even if the largest consumer chatbots do not yet. If your audience is developers who use those tools, the file has a real, if narrow, audience today.
- It forces a useful exercise. Writing an
llms.txtfile means auditing what your most important pages actually are. That clarity helps your broader AI-search and structured-data work even if the file itself goes unread by ChatGPT.
The realistic framing: llms.txt is a low-cost, speculative bet on a standard that has not been adopted by the companies that would make it matter most, not a proven AI-visibility tactic. If you have ten minutes and no urgent competing priority, add one. If you are choosing between an llms.txt file and fixing your actual content structure, schema, and author attribution, do the second one first, it is the part with demonstrated effect on AI citations.
How llms.txt Differs From robots.txt and sitemap.xml
They live in the same root directory and share a plain-text spirit, but they do three different jobs.
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Audience | Search-engine crawlers | Search-engine crawlers | AI models / LLM-based tools |
| Purpose | Block or allow crawling of paths | List every indexable URL for discovery | Curate and summarize the pages that matter most |
| Enforced by | Yes, honored as a de facto standard by major search engines | Yes, submitted via Search Console and read by crawlers | No, not officially adopted by major AI providers yet |
| Format | Plain text, directive syntax | XML | Markdown |
| What happens if missing | Crawlers assume everything is allowed | Crawlers fall back to link discovery | Nothing breaks, this is purely additive |
The practical takeaway: robots.txt restricts, sitemap.xml lists everything, and llms.txt curates a subset with context. None of them replace the others, and skipping llms.txt costs you nothing measurable today.
How to Set Up llms.txt: Step by Step
- List your five to fifteen most important pages. Not everything, the pages you would want a model to read first if it could only read a handful: your core product explanation, key docs, pricing, and your best evergreen guides.
- Write a one- or two-sentence summary of your site. This goes in the blockquote under your H1. Say what the site is plainly, skip the marketing adjectives; a model extracts facts better than tone.
- Group your links under two or three H2 sections. Common groupings: Docs, Guides, Product, Company. Each link gets a short, factual description, not a slogan.
- Save the file as
llms.txtin your site's root directory. For most CMS platforms this means the same location asrobots.txt: the public root, not a subfolder. - Validate it loads at
yoursite.com/llms.txt. Check it renders as plain text (not wrapped in your site's HTML template) and returns a 200 status. - Revisit it when your content map changes. Treat it like a sitemap: update it when you launch major new docs or retire an old page, not on a fixed schedule.
If you are on WordPress, Webflow, or another CMS without native support, the simplest path is a plugin or a manually uploaded static file at the root; if you cannot write directly to the root for platform reasons, an llms.txt is not worth a major infrastructure workaround given the current adoption reality above.
Where llms.txt Fits Into a Real AI-Search Strategy
An llms.txt file is a small, additive piece of a much bigger picture. The signals that actually move whether AI engines cite you, structured content, clear author attribution, current data, and clean technical fundamentals, matter far more today than any one file at your root. Murkuz's AI search readiness use case grades every page A through F against exactly those factors (structure, author authority, freshness, source attribution, topical depth, and technical signals) and generates the specific fix for each page, rather than betting on one file that most AI crawlers still ignore.
For a site publishing a lot of structured reference content, pairing an llms.txt file with proper schema markup covers both audiences: search engines that read structured data, and the smaller set of AI tools that already read llms.txt. Murkuz's features page details how its AI readiness scoring and structured-content recommendations work together on a single page.
FAQ
What does an llms.txt file do?
It gives AI models and AI-aware tools a short, curated summary of your site plus links to your most important pages, so a model does not have to guess which parts of your site matter most. It does not block or allow crawling the way robots.txt does.
Does llms.txt actually work?
Not for the largest AI providers yet. As of mid-2026, OpenAI, Anthropic, Google, and Meta have not confirmed they read or weight llms.txt files for citations, and independent research has found most published llms.txt files are never requested by any bot. Some AI-native developer tools do read it. Treat it as low-cost and speculative, not proven.
What is the difference between robots.txt and llms.txt?
robots.txt tells search-engine crawlers what they may or may not crawl, and is honored as a de facto standard. llms.txt curates a summary and a shortlist of key pages for AI models to read, and is not officially adopted by any major AI provider. One restricts, the other suggests.
How do I write an llms.txt file?
Start with an H1 for your site name, a blockquote with a one-line summary, then one or more H2 sections grouping markdown links to your most important pages with short descriptions. Save it as llms.txt at your site's root, the same location as robots.txt.
Where do I put my llms.txt file?
At your site's root directory, so it loads at yoursite.com/llms.txt, the same location convention as robots.txt and sitemap.xml.
I have run SEO as the first growth channel across several of my own products, and the pattern with new standards like this one is always the same: the low-cost, low-risk version is worth doing on a slow afternoon, but it is never a substitute for the fundamentals that are actually proven to move AI citations. Ship the llms.txt file if you have ten minutes, then spend the rest of your time on structure, authorship, and freshness, the things that demonstrably work.
Junaid Khalid, Founder of Ertiqah (builder of Murkuz)
