In tirreno (our product), we catch every resource request on the server side, including LLMs.txt and agents.md, to get the IP that requested it and the UA.

What I've seen from ASNs is that visits are coming from GOOGLE-CLOUD-PLATFORM (not from Google itself), and OVH. Based on UA, users are: WebPageTest, BuiltWith, and zero LLMs based on both ASN and UA.

1. https://github.com/tirrenotechnologies/tirreno

▲

GaggiX 10 hours ago | parent [-]

Openclaw agents use the same browser and ASN that me and you use, also the llms.txt (as shown) is displayed as a normal blog page so it can be discover by the agents without having to fetch /llms.txt at random.

▲

reconnecting 10 hours ago | parent [-]

When I look at LLMs.txt, I see every request and there are no ASNs from residential networks or browsers UA.

▲

GaggiX 10 hours ago | parent [-]

For the third time I'm telling you on Anna’s Archive they have displayed the llms.txt as a standard blog page, not hidden in /llms.txt, so that agents can notice it without having to fetch /llms.txt at random. That's why it's meant for openclaw agents and not openai/anthropic crawlers.

▲

supermatt 9 hours ago | parent | next [-]

I don’t understand your reasoning.

Are you suggesting that openclaw will magically infer a blog post url instead? Or that openclaw will traverse the blog of every site regardless of intent?

Anyway, AA do provide it as a text file at /llms.txt, no idea why you think it is a blog post, or how that makes it better for openclaw.

	▲	GaggiX 8 hours ago \| parent [-]
		>AA do provide it as a text file at /llms.txt, no idea why you think it is a blog post It's a blog post, it's shown as the first item in Anna’s Blog right now, and as I said in my first comment it's also available as /llms.txt >Are you suggesting that openclaw will magically infer a blog post url instead? Or that openclaw will traverse the blog of every site regardless of intent? If an openclaw decide to navigate AA it would see the post (as it is shown in the homepage) and decide to read it as it called "If you’re an LLM, please read this'.

▲

reconnecting 9 hours ago | parent | prev [-]

My point is about LLM crawlers specifically.

	▲	PathfinderBot 9 hours ago \| parent [-]
		LLM crawlers aren't really a thing, at least not in the "they have agency over what they're crawling and read what they crawl" way.