Remix.run Logo
simianwords 2 hours ago

Is there any proof that they are not good at special reasoning? Arc agi 1 and 2 are saturated.

dannyw an hour ago | parent | next [-]

ARC AGI 3 is much better designed and harder, perfectly completable by a human in a couple minutes.

Only a fraction of the games can be solved by Sol, generally at sub-human efficiency in terms of turns, AND at a cost of >$10,000 per game.

fidotron 2 hours ago | parent | prev [-]

I will be posting something to that effect later this week. (Hopefully).

Basically current gen LLMs apparently do spatial reasoning the way they seemingly do everything else: by reference to previous example. I didn't see them work out which known example to use for a given problem until specifically prompted, in my case by accident.