Websites should have a lean markdown or .txt page for each human friendly webpage. A lot of the surge in bots is because of LLMs. Its insane that a technical documentation web page can use 200MB + of memory, when the core information I care about is << 1 MB of text. at the path of least resistance for many people is to have claude code hit 20 of such pages.

This is something that would be perfect for cloudflare to host and sell as a service - static web pages via their CDN network.

I do not work in web development, so im sure there are plenty of details im ignorant of, but the TLDR of "how to fight accidental DDOS because of AI tooling " is make it easier for them to get the content they want.