I've always browsed with javascript disabled but in the last few months (presumably in response to AI scraping) loads of sites that previously worked now don't. IMDB. Loads of open-source blogs, wikis and source repositories. Commenting on Wikipedia. Browsing job sites.

It's never been easier to create a great site that doesn't require javascript, but hardly anyone is.

Google Search doesn't work without JS. I think we're actually moving in the opposite direction of what OP inferred. It's pretty difficult to reliably detect bots without using JS, and the vast majority of interesting client-side web applications are downright impossible without it. No amount of HATEOAS is going to make a usable version of Figma.

I think there’s a lot of good reasons to, but hardly any incentive to.

People who disable JS are probably a very tiny minority and of those who consume ads, an even smaller one.