I agree it's not a good idea, but:

> Remember this is the same information that every regular user is able to use.

This is not true. The users don't see the HTML, CSS and JavaScript. If they would, even if they're frontend devs, they'll be just as confused as the agents.

The user see images, and while you can make the agents see the image as well, unlike humans the effective agents we have today work with text, not images.

To clarify, I am referring to the information present on the webpage which the users using the website are already consuming. The page already has a structure, thats how browser is able to display it to end user. Its just not tailored explicitly to a agent.

Lets leave the image vs text from the discussion from now, its a altogether different challenge.