I'm working on a WordPress PaaS with dedicated lanes for bots. The status quo around WordPress is that you block bots using Cloudflare, else your site crashes. Since AI search is here to stay, we need a way to let bots crawl WordPress sites without crashing the server.

Currently at MVP stage, no domain yet.

Regular Cloudflare + heavy caching should solve all crawling problems, no?

For most bot visits, there should not be a single database request.