The llms.txt Myth: 10 Million AI Requests Analyzed, 0 Reads (Senthor Study)

The llms.txt Hype

For the past few months, the llms.txt file has been presented as THE miracle solution to give instructions to artificial intelligences. Like a robots.txt for the future, where you could tell AIs what they can or cannot scrape.

Dozens of blog posts, Twitter threads, and Hacker News discussions praise this new "standard". Some developers have even already added it to their site.

It looks good on paper, but do robots actually read it?

At Senthor, we don't make assumptions. We checked the server logs. Here's the truth.

The Senthor Study: 10 Million vs Zero

Since September 2025, Senthor has been analyzing traffic from thousands of websites to identify AI bots. We have a privileged view of how major models (OpenAI, Anthropic, Google, Perplexity) actually interact with websites.

Here are the numbers:

Translation: Out of 10 million confirmed visits from major AI bots, exactly zero attempted to read the llms.txt file. The 104 detected requests came from curious humans or small unidentified scrapers, not the LLMs you're trying to control.

Live Proof

This data was publicly shared by Matt El mouktafi, CTO of Senthor, on LinkedIn:

Raw data shared by Matt El mouktafi, CTO of Senthor.

Why Doesn't It Work?

There are two major technical reasons why the llms.txt file is ignored:

1. No Official Standard

OpenAI, Google, and Anthropic have never officially adopted this standard. The llms.txt file is a community invention that has no support from major players.

These companies follow (sometimes) robots.txt because it's a 30-year-old historical convention. But they have no obligation or intention to read recent community inventions.

2. AIs Don't Index Websites

Unlike traditional search engines (Google, Bing), AIs don't crawl your site exhaustively. That's not their job.

AI assistants (ChatGPT, Claude, Perplexity) use existing search systems to find relevant information. When a user asks a question, the AI queries a search engine, identifies the specific page containing the answer, and visits only that page. All in milliseconds.

They won't consult your sitemap.xml or llms.txt to browse your site looking for information. They access the target page directly via search, extract the content, and leave.

Figure: AI traffic analysis on Senthor - No llms.txt reads detected

How to Actually Control AIs in 2026?

Stop believing in magic solutions. Here's what actually works:

✅ Tip 1: Keep Your robots.txt

It's the "bare minimum". Some bots still respect it (not all, but it's better than nothing). As we've seen in our article on why robots.txt is no longer enough in the AI era, it's not absolute protection, but it's still the baseline.

✅ Tip 2: Server-Side Detection

Since AIs don't ask permission (don't read llms.txt), you need to detect them Server-Side. You must analyze their signature, User-Agent, IP, and behavior to identify them before they scrape your content.

That's exactly what Senthor does: identify AI bots in real-time, block them if you wish, or measure their impact. As we showed in our analysis of Dark AI Traffic in Google Analytics, server-side detection is the only reliable method.

✅ Tip 3: Optimize for Answers, Not Files

If you want AIs to cite your content correctly, optimize your structured content (GEO - Generative Engine Optimization). Use visibility tools like Atyla to check how AIs understand your site, not invisible text files that no one reads.

Conclusion: Stop Flying Blind

The llms.txt file is a beautiful idea in theory. But data doesn't lie.

Out of 10 million certified AI requests, zero major bots attempted to read it. That's a fact, not an opinion.

If you really want to control how AIs interact with your content, base yourself on technical reality, not community hype.