Does Adding an llms.txt File Help Your Website?

I added an llms.txt file to matthewjohn.design when the discussion started picking up. I wanted to understand it from the inside rather than just summarise what other people were saying about it — so this post is based on that experience, plus the research I did along the way.

The short answer: it almost certainly isn’t helping your search or AI visibility right now. But the reasons why are interesting, the situation is evolving faster than most coverage suggests, and there’s at least one genuinely useful reason to have it that most posts about this topic miss entirely.

What is llms.txt?

If you know what robots.txt does, the concept is familiar. robots.txt tells search engine crawlers what they’re allowed to index. llms.txt is an unofficial proposal — put forward by Jeremy Howard of Answer.AI in September 2024 — designed to do something similar for AI models.

The idea: place a plain-text Markdown file at yourdomain.com/llms.txt that points AI models toward your most important pages, explains what your site is about, and potentially signals how you want your content to be used. Unlike robots.txt, it has no formal standard, no enforcement mechanism, and — until recently — no major AI provider had announced support for it.

What the data actually shows

Your original instinct to be sceptical was right. The evidence is fairly clear that llms.txt has no measurable impact on AI visibility right now.

A structured analysis of 300,000 domains by SE Ranking found no correlation between having an llms.txt file and how often a domain gets cited by AI tools — with 8 out of 9 sites in their test showing no measurable change after implementation. Independent server log audits confirm why: GPTBot, ClaudeBot, PerplexityBot, and Google-Extended consistently don’t request the file. If a crawler isn’t fetching it, it can’t possibly influence anything.

Google’s own engineers have been direct about this. John Mueller has stated publicly that no AI systems currently use llms.txt, and that you can verify this in your own server logs — the requests simply aren’t there. Google continues to rely on traditional SEO signals for AI Overviews and other AI-powered features, not a separate text file.

Adoption has grown significantly — from around 0.3% of domains in early 2025 to roughly 10% by early 2026 — but high adoption doesn’t mean high effectiveness. It mostly means the topic got a lot of SEO blog coverage.

The one use case that genuinely works right now

Here’s what most posts on this topic miss: AI coding assistants already use llms.txt.

Tools like Cursor and Claude Code — and MCP (Model Context Protocol) servers more broadly — do actively fetch llms.txt files when they’re trying to understand a project or codebase. For developer-focused sites, documentation, or any site that serves a technical audience, this is a real reason to have one. It’s not SEO, it’s context-setting for the AI tools that your audience is actually using.

It’s also worth noting that Anthropic itself has published an llms.txt on their own website — which at minimum suggests the idea isn’t going anywhere, even if the broader AI crawler adoption hasn’t materialised yet.

Should you add it?

Yes — but with realistic expectations.

The case for adding it isn’t that it will boost your rankings or AI citations today. It’s that:

• It takes about 10 minutes to set up on Webflow
• It documents your content hierarchy clearly, which is useful regardless of whether AI crawlers read it
• AI coding tools and MCP servers do use it, so if any of your audience uses those tools to research your space, you’re already set up
• If major AI providers do adopt it — which remains plausible — you’re ahead of the curve rather than scrambling to catch up

The case against is basically just opportunity cost. If you’re choosing between adding llms.txt and improving your actual content quality, structured data, or internal linking — do those things first. Those are the signals AI tools demonstrably use.

What actually moves the needle for AI visibility

AI Overviews, ChatGPT citations, and Perplexity results all draw from the same well as traditional search: well-structured, authoritative content on a site Google already trusts. The same things that help SEO help AI visibility — with a few nuances:

• Clear, direct answers near the top of the page. AI tools tend to surface content that answers a specific question concisely, without requiring the model to parse through three paragraphs of preamble.
• Structured data and schema markup. Increases the chance of being pulled into rich results and AI summaries.
• Allowing AI crawlers in robots.txt. OpenAI recommends explicitly allowing GPTBot. If you’ve blanket-blocked all bots, you’ve blocked AI crawlers too.
• Domain authority. AI tools lean heavily on sources they’ve been trained to trust. Building a reputation in your niche over time is still the most reliable route to AI citation.

I saw 190 visitors to matthewjohn.design from chatgpt.com in a recent 90-day period, with a relatively low bounce rate — that traffic came from content quality and domain signals, not from llms.txt.

How to add llms.txt in Webflow

Webflow has built-in support for llms.txt. Go to Site Settings → SEO → llms.txt, create a plain-text Markdown file named llms.txt, and upload it there. It’ll be served at yourdomain.com/llms.txt.

A useful llms.txt for a small business site looks something like this:

# Matthew John Design
> Webflow-exclusive web design studio for service businesses.

## Services
- [Web Design](/services): Webflow design and development
- [Support Retainers](/support): Ongoing Webflow maintenance

## Best posts
- [How to Speed Up Your Webflow Site](/article/how-to-speed-up-your-webflow-site)
- [Is Webflow Good for SEO?](/article/is-webflow-good-for-seo)

Keep it focused on your most important pages. This isn’t a sitemap — it’s a curated summary of what your site is and what matters most.

Bottom line

llms.txt is an interesting early-stage standard that doesn’t do much for most sites right now, but costs almost nothing to add and has at least one genuinely active use case in AI coding tools. Add it, write it properly, and then put your energy into the things that demonstrably work: content quality, structured data, and making sure you’re not accidentally blocking AI crawlers in robots.txt.

I’ll update this post as the situation develops — it’s moving faster than almost any other area of web practice right now.

Thinking about your site’s visibility in AI search?

Matthew John Design builds Webflow sites with SEO and AI visibility built in from the start — structured content, clean code, and proper technical setup. Get in touch to talk about your project.