Why would that happen if the website owners are explicitly trying to prevent access? This isn't a technical issue, the agents are perfectly capable of working with the existing websites regardless of ads and javascript. The semantic web and machine readable representations was a 90s thing, agents don't need that really.

the js/yaml version would require a license key that connects to a business arrangement between the website and the owner of the crawler. the plain HTML version would adversarially block them (or even identify them, where they could be sued...once we have legal precedents for this stuff).