I think the short answer, is SEO is dying rapidly in the age of AI search engines. Search traffic is anecdotally being severely damaged when everyone can just get an AI answer; ad revenue is down especially after the pandemic; SEO spam is increasing from AI-generated websites.

We might reach the point of saying, screw it, it hardly matters anymore. The web has been conquered by LOB apps, Paywalls, and Walled Gardens; and there’s no real way to fix that, but there is a way to fix the tech we are using.

If anything, if we fully embrace “web as app engine,” we can fix some of these issues ahead of time. Maybe we have a standard entry point for crawlers (or users) requesting the raw text of a document. It’s better than being stuck in this awful in-between which we are right now.