When I made a computer use tool on macOS, my tool could find the right button easily, because well made apps have buttons with a role and a name. But I made my tool find the right button using screenshots at bad apps.