Remix.run Logo
eugene3306 8 hours ago

why don't they publish at ARC-AGI ? too expensive?

Bolwin 7 hours ago | parent [-]

Arc agi was never a good benchmark that tested spatial understanding more than reasoning. I'm glad it's no longer popular

falcor84 7 hours ago | parent | next [-]

What do you mean? It definitely tests reasoning as well, and if anything, I expect spatial and embodied reasoning to become more important in the coming years, as AI agents will be expected to take on more real world tasks.

eugene3306 7 hours ago | parent | prev [-]

spatial or not, arc-agi is the only test that correlates to my impression with my coding requests