Remix.run Logo
lxe 2 days ago

Before I dive into the source code... how do you pass the page content, and the locations of interactive components to the LLM? And how do you dispatch events to interact with the page? I just want to verify if it's ARIA tree like the others, or it's something else.

felarof 2 days ago | parent [-]

Today, we connect to chrome using CDP and use Puppeteer to send clicks and other operations. Also, using browser use DOM tree highlighting, which works great.

To get the page content we parse accessibility tree.