Interesting that using tools to zoom around the image is useful for the model. I was kind of assuming that these models were beyond such things and could attend to all aspects image simultaneously anyway, but perhaps their input is still limited in the resolution? Very cool, in any case, spooky progress as always.

There's just a certain amount of things the image encoder can process at once. It's pretty apparent when you give the models a big table in an image.