Remix.run Logo
▲ _heimdall 18 hours ago

Yep, I ran into similar concerns around 15 years ago while working for Windows Phone, though at the time we didn't actually expose accessibility tree access.

I both don't understand why LLM providers want agentic use via screenshots or screen scraping and am grateful that they didn't decide to just use browser accessibility trees for browser use features. I don't expect we'll be that lucky on mobile where a11y features don't require industry agreement and standard implementations.