I added in the system prompt that it should skip CAPTCHAs and hand control back to the user. Currently working on a proper human-in-the-loop feature. That's actually one of the key advantages of running the agent inside your own browser.

the CAPTCHA question points to a deeper issue: even before the CAPTCHA, most sites are already fingerprinting the browser that's running the agent. if the underlying browser leaks automation signals - navigator.webdriver, Canvas fingerprint deviations, WebGL anomalies - the session gets flagged or soft-blocked long before a CAPTCHA is even served.

the "inside your own browser" angle is actually the right intuition here. a real user's browser has built up a consistent fingerprint profile across sessions. the moment you run an agent in a context where those signals differ from that baseline, you're detectable. curious whether you've run into this on sites with aggressive bot detection, or whether the use case has mostly been internal/enterprise apps where that's not a concern?

Makes sense.

For curiosity's sake, have you had it try to attempt captchas?

If so, what were the results?