When I look at LLMs.txt, I see every request and there are no ASNs from residential networks or browsers UA.

For the third time I'm telling you on Anna’s Archive they have displayed the llms.txt as a standard blog page, not hidden in /llms.txt, so that agents can notice it without having to fetch /llms.txt at random. That's why it's meant for openclaw agents and not openai/anthropic crawlers.

▲

supermatt 9 hours ago | parent | next [-]

I don’t understand your reasoning.

Are you suggesting that openclaw will magically infer a blog post url instead? Or that openclaw will traverse the blog of every site regardless of intent?

Anyway, AA do provide it as a text file at /llms.txt, no idea why you think it is a blog post, or how that makes it better for openclaw.

	▲	GaggiX 8 hours ago \| parent [-]
		>AA do provide it as a text file at /llms.txt, no idea why you think it is a blog post It's a blog post, it's shown as the first item in Anna’s Blog right now, and as I said in my first comment it's also available as /llms.txt >Are you suggesting that openclaw will magically infer a blog post url instead? Or that openclaw will traverse the blog of every site regardless of intent? If an openclaw decide to navigate AA it would see the post (as it is shown in the homepage) and decide to read it as it called "If you’re an LLM, please read this'.

▲

reconnecting 9 hours ago | parent | prev [-]

My point is about LLM crawlers specifically.

	▲	PathfinderBot 9 hours ago \| parent [-]
		LLM crawlers aren't really a thing, at least not in the "they have agency over what they're crawling and read what they crawl" way.