The File Your Website Is Missing: What llms.txt Actually Does

Search engines used to be the only audience your website needed to write for. That’s no longer true. A growing share of the traffic hitting your pages today isn’t a person clicking a blue link — it’s a language model reading your site to answer someone else’s question. And most sites still haven’t adjusted anything for that reader.

One of the simplest, most overlooked adjustments is a small text file called llms.txt. It’s not a ranking trick and it won’t replace good SEO work, but it solves a real problem: AI crawlers land on your site with no map, no context, and a limited budget of pages they’ll actually process. llms.txt gives them one.

What llms.txt actually is

llms.txt is a plain Markdown file, served at the root of your domain (yoursite.com/llms.txt), that summarizes what your site is, what it offers, and which pages matter most. Think of it as an executive briefing written for a machine instead of a human: no marketing copy, no fluff — just a clear map of your most important content, in the format a language model can parse in one pass.

It was proposed as an open convention (not an official web standard yet, and not controlled by any single company), but it’s already being read by several AI tools and crawlers, and adoption is growing the same way sitemap.xml adoption grew a decade ago: quietly, then everywhere.

robots.txt, sitemap.xml, and llms.txt: three different jobs

These three files are often confused because they all live at the root of your domain, but each answers a different question.

	robots.txt	sitemap.xml	llms.txt
Answers	What can you crawl?	What pages exist?	What actually matters, and what does it mean?
Audience	Search engine crawlers	Search engine indexers	Language models and AI assistants
Format	Rules (allow/disallow)	XML list of URLs	Human-readable Markdown summary
Optional?	Recommended	Recommended	Optional, emerging practice

None of the three replaces the others. A site with solid technical SEO already has the first two. llms.txt is the missing piece for the audience that doesn’t crawl your whole site the way Googlebot does — it samples, summarizes, and moves on.

Why this matters now

AI assistants answer questions by pulling context from the web in real time or by having been trained on it. Either way, they work with limited context windows — they can’t read your entire site the way a human researcher would. If your most important information is buried three clicks deep, split across ten pages, or wrapped in JavaScript that never renders in a lightweight fetch, it simply doesn’t make it into the answer.

llms.txt doesn’t fix bad content or a confusing site structure. What it does is remove the guesswork: instead of hoping a crawler stumbles onto your pricing page or your documentation, you point directly at it, with a one-line description that tells the model why it matters. It’s a small file that compensates for a real constraint on how these systems read the web today.

What a good llms.txt file contains

The convention is intentionally simple: an H1 with your site or company name, a one-line summary, and a series of H2 sections linking to grouped pages. In practice, a useful llms.txt tends to include:

• A short, factual description of what the company does — no adjectives, no slogans.
• Direct links to the pages that actually answer buying questions: services, pricing logic, case studies, documentation.
• A one-line description next to each link, explaining what the linked page covers.
• An “optional” section for secondary content (blog, changelog, legal pages) that’s useful but not essential.

A minimal example looks like this:

# Codexia

> Custom web development and applied AI systems for companies, based in Argentina.

## Main pages

- [Services](https://codexia.com.ar/services): custom web development, API-first architecture, and AI automation.
- [Blog](https://codexia.com.ar/blog): technical articles on web architecture, automation, and AI implementation.
- [Contact](https://codexia.com.ar/contact): how to reach the team and start a project.

## Optional

- [Legal](https://codexia.com.ar/legal): terms and privacy policy.

Notice what’s missing: no keyword stuffing, no calls to action, no design. It’s written to be parsed, not persuasive.

How to add it to a WordPress site without breaking anything

On WordPress specifically, there are two safe ways to do this:

1. Static file at the root: the cleanest option. Upload llms.txt directly to the site’s root directory via SFTP so it’s served at yoursite.com/llms.txt, exactly like robots.txt. No plugin, no database, nothing that can be broken by a theme update.
2. Rewrite rule: if you don’t have file system access, a rewrite rule that maps /llms.txt to a page or a virtual endpoint works too, though it adds one more moving part to maintain.

Either way, keep it in version control or in your deployment checklist. It’s a file that goes stale quietly — nobody notices a broken link inside a file almost no human ever opens, which is exactly why it’s worth reviewing every time the site’s main pages change.

Common mistakes we see

• Treating it like meta keywords 2.0. Stuffing it with target terms doesn’t work the way old-school SEO tricks did, and it undermines the one thing that makes the file useful: being trustworthy and specific.
• Linking to pages that require login or JavaScript rendering. If the linked page can’t be fetched and read as plain text, the entry is close to useless.
• Writing it once and forgetting it. A llms.txt pointing at a discontinued product or a 404’d page actively hurts more than having no file at all.
• Expecting it to fix a content problem. If the linked pages are thin, outdated, or contradictory, no summary file will make an AI system recommend you.

FAQ

Is llms.txt an official standard?

No. It’s a community-proposed convention, similar to how sitemap.xml started before search engines formally adopted it. Support is inconsistent across AI tools today, but the direction is clear enough that adding it costs almost nothing and closes a real gap.

Will this improve my ranking on Google?

It’s not built for that, and there’s no evidence it affects traditional search rankings. Its purpose is helping AI assistants and language models understand and cite your site correctly — a different, complementary goal to classic SEO.

Do I need a plugin to add it on WordPress?

No. A single static file uploaded to the site’s root is enough, and it’s the more reliable option since it doesn’t depend on the plugin ecosystem or theme changes.

How often should I update it?

Whenever your main pages, services, or pricing structure change meaningfully — the same trigger you’d use to review your sitemap.

None of this replaces a solid technical foundation: fast pages, clean HTML, real content behind every link. llms.txt is a small, low-effort addition on top of that foundation — one more detail we build into the sites we develop, alongside the sitemap and the schema markup, because ignoring how machines read a website is no longer an option, whether that machine is a search engine or an AI assistant.