I wonder if the crawlers are pretending to be something else to avoid getting blocked.

I see Bun (which was bought by Anthropic) has all its documentation in llms.txt[0]. They should know if Claude uses it or wouldn't waste the effort in building this.

[0] https://bun.sh/llms.txt

As a project that started with a lot of idealism about how software _should_ be built, I would totally expect Bun to have an llms.txt file even if Claude wasn't using it. It's a project that is motivated in part by leading by example.

Did they do that before they were bought by Anthropic? Perhaps it's just part of a CI process that nobody's going to take an axe to without good reason.

I also noticed this LLMs.txt at bun.sh, so for me it looks like some sort of advertising.