Remix.run Logo
jonplackett 10 days ago

I just tried a few things that are simple and a world model would probably get right. Eg

Question to GPT5: I am looking straight on to some objects. Looking parallel to the ground.

In front of me I have a milk bottle, to the right of that is a Coca-Cola bottle. To the right of that is a glass of water. And to the right of that there’s a cherry. Behind the cherry there’s a cactus and to the left of that there’s a peanut. Everything is spaced evenly. Can I see the peanut?

Answer (after choosing thinking mode)

No. The cactus is directly behind the cherry (front row order: milk, Coke, water, cherry). “To the left of that” puts the peanut behind the glass of water. Since you’re looking straight on, the glass sits in front and occludes the peanut.

It doesn’t consider transparency until you mention it, then apologises and says it didn’t think of transparency

wilg 8 days ago | parent | next [-]

Worked for me: https://chatgpt.com/share/689bc3ef-fa1c-800f-9275-93c2dbc11b...

optimalsolver 8 days ago | parent | prev | next [-]

Gemini 2.5 Pro gets this correct on the first attempt, and specifically points out the transparency of the glass of water.

https://g.co/gemini/share/362506056ddb

Time to get the ol' goalpost-moving gloves out.

RugnirViking 10 days ago | parent | prev [-]

this seems like a strange riddle. In my mind I was thinking that regardless of the glass, all of the objects can be seen (due to perspective, and also the fact you mentioned the locations, meaning you're aware of them).

It seems to me it would only actually work in an orthographic perspective, which is not how our reality works

jonplackett 9 days ago | parent [-]

You can tell from the response it does understand the riddle just fine, it just gets it wrong.

rpdillon 6 days ago | parent | next [-]

I'm not sure it does. I did ask 5 adults this question, with zero context about what we're discussing with AI, just posing it as a riddle. They were split, with lots of uncertainty about the optics of the glass straight on, and where the viewer is vertically with respect the glass' height. The best response, from my wife, brought up the trig aspect to the problem, pointing out that nothing in the question talks about distance to or between the objects. Her assumption was that the peanut could easily be offset behind the glass from her perspective, resulting in the peanut not being visible.

We tried the experiment, and she was right that there are definitely setups of distance and spacing that cause the peanut to not be visible. Try it!

rpdillon 8 days ago | parent | prev [-]

Have you asked five adults this riddle? I suspect at least two of them would get it wrong or have some uncertainty about whether or not the peanut was visible.

xg15 8 days ago | parent [-]

This. Was also thinking "yes" first because of the glass of water, transparency, etc, but then got unsure: The objects might be spaced so widely that the milk or coke bottle would obscure the view due to perspective - or the peanut would simply end up outside the viewer's field of vision.

Shows that even if you have a world model, it might not be the right one.