New technologies ·
What Is llms.txt and Does Your Website Need One?
By Manu Mathew, Technical Lead Engineer · Last updated 7 October 2026
If you’ve opened your SEO plugin’s settings recently, you may have seen a new option called llms.txt. Some articles call it essential for AI search. Others say it does nothing.
Both views miss the point a little. An llms.txt file is a small, optional guide for AI tools, and whether it’s worth having depends on what those tools need from your site.
Quick answer
An llms.txt file is a short Markdown file at the root of a website that gives AI assistants a summary of the site and links to its most useful pages. Google says its Search ignores the file, so it won’t lift rankings. It’s easy to add and may help some AI tools, but good pages matter far more.
What is an llms.txt file?
It’s a plain text file, written in Markdown, that sits at yoursite.com/llms.txt and summarises your site for AI tools.
The idea comes from a public proposal at llmstxt.org, first published in 2024 and revised in August 2026. Web pages wrap their content in menus, scripts and adverts, and an AI assistant has limited room to read. A short, clean overview helps it find the right page faster.
The format is simple:
- A title. One heading with the name of the site. This is the only required part.
- A short summary. A line or two saying what the site is and who it’s for.
- Optional notes. Anything that helps a reader understand the site.
- Lists of links. Grouped under headings, each with a short description.
Think of it as the board at a shop entrance showing what’s on each floor, rather than a list of every item on every shelf.
How is llms.txt different from robots.txt and a sitemap?
They do three different jobs: robots.txt sets rules for crawlers, a sitemap lists every page, and llms.txt points AI tools to the pages that matter most.
| File | What it’s for | Who reads it |
|---|---|---|
| robots.txt | Says which parts of the site crawlers may visit | Search engines and other bots |
| sitemap.xml | Lists every page you want indexed, with update dates | Search engines |
| llms.txt | A short, hand-picked guide to your key pages | AI assistants and agents, if they choose to |
The proposal itself says llms.txt doesn’t replace either file. A sitemap is usually too long for an AI tool to read in one go, and robots.txt says nothing about what your pages are about.
Does llms.txt help SEO or AI Overviews?
Not on Google. Google says its Search, including AI Overviews and AI Mode, doesn’t use llms.txt files at all.
Google’s guide to generative AI features in Search lists llms.txt among the things you can ignore. It says the file will neither harm nor help your visibility, because Search ignores it. What helps is the usual work: useful, original pages that can be crawled and indexed.
I made the same point about whether AI-generated content hurts SEO: there’s no shortcut around helpful pages. Other AI services don’t say clearly whether they read llms.txt, so for a typical small business site the benefit is uncertain.
Why does Lighthouse check for llms.txt?
Chrome’s Lighthouse now has an experimental “Agentic Browsing” section that checks how easy a site is for AI agents to use. A missing file isn’t counted against you.
Lighthouse’s llms.txt audit marks a missing file as not applicable, because the file is optional. It only fails when the server returns an error, or when a file exists but doesn’t follow the basic format.
So if you see this check in a PageSpeed report, treat it as a hint rather than a problem. The same section also looks at labels on buttons and forms and at layout stability, which help human visitors too.
Does your website need an llms.txt file?
Most small business sites don’t need one, but a good one does no harm. It’s most useful when a site has documentation or lots of reference content.
Here’s how I’d decide:
- You run a software product or API. Worth adding. Coding assistants often look for it in documentation.
- You have a large site with guides or resources. Worth considering, to point AI tools to your best pages.
- You have a small brochure site. Optional. Your pages, contact details and Google Business Profile matter more.
- Your plugin already makes one. Check that it reads well, then leave it on or switch it off.
If your main pages are thin or out of date, fix those first. An llms.txt file can only point to what’s already there.
How do you add llms.txt to a WordPress site?
The simplest way is through your SEO plugin, as both Yoast SEO and All in One SEO can now generate the file for you.
Turn it on in the plugin’s settings, then open yoursite.com/llms.txt in a browser and read it as a stranger would. Check that:
- The summary is accurate. Your business name, what you do and where you work.
- The links point to your key pages. Services, contact, about and your most helpful guides.
- Nothing private is listed. No test pages, old drafts or pages you’ve hidden from search.
- It loads quickly. A file rebuilt on every visit can be slow, so it’s worth caching.
If you’d rather write it by hand, a plain text file uploaded to the site’s root folder works too. Keep it short, and update it when your main pages change.
How do I handle llms.txt for clients?
I treat it as a small, optional extra, never as a ranking promise. It comes after the pages themselves are clear, accurate and easy to find.
When I set up a WordPress site, the order is:
- Pages first. Clear services, contact details and answers to real customer questions.
- Search basics. Titles, descriptions, a sitemap and a correct robots.txt.
- Then llms.txt, if it fits. Short, accurate and limited to pages the owner is happy to point people to.
- A written note. I explain what the file does and doesn’t do, so nobody expects a jump in traffic.
This site has a short llms.txt that I keep in step with its sitemap. It takes little effort to maintain, and I don’t count on it for visitors. For the kinds of WordPress work I take on, see the WordPress websites page.
Frequently asked questions
Is llms.txt an official web standard?
No, it’s a public proposal, not a standard from a body such as the W3C or the IETF. Many sites and tools have adopted it, including popular WordPress SEO plugins, but each AI service decides for itself whether to read it. Treat it as a useful convention rather than a rule.
Can llms.txt stop AI tools from using my content?
No, an llms.txt file only suggests where to look and doesn’t block anything. To ask crawlers not to visit parts of your site, use robots.txt rules instead. Well-behaved bots follow those rules, but badly behaved ones may ignore them.
What is llms-full.txt?
It’s a larger companion file that some tools, including SEO plugins, can generate. Instead of links, it holds the full text of your pages in Markdown. It can suit documentation sites, but for a small business site it’s usually unnecessary, and the short llms.txt is enough.
Will adding llms.txt slow down my website?
No, the file is only fetched when a tool asks for it, so ordinary visitors never load it. The one thing to watch is a plugin that rebuilds the file on every request, which can make the file itself slow to load. Caching it fixes that.
Should I update llms.txt when I add new pages?
Only when the new page is one of your key pages. The file is meant to be a short guide, not a full list, so adding every blog post defeats the purpose. Plugin-generated files often update themselves, but it’s still worth checking them every few months.
Want help making your site easy to find?
Whether you need a new website or help with the one you have, I’m happy to talk it through. That includes the search and AI basics, and an honest view of which extras are worth your time. I work with small businesses in Kozhikode, across Kerala and remotely.
You can reach me through the contact section or email me at manu@manu.co.in. Tell me a little about your business and the site, and we’ll go from there.