LLMs really don't understand space.

Yesterday I was trying to add a patio to some real estate photos, i.e. "virtual staging".

I uploaded 4 different photos of the same house, from different angles.

I asked Gemini, ChatGPT, and Grok add the same patio to each photo. The results were laughably incoherent (not to mention ugly).

I repeatedly asked them to make the 4 photos consistent in 3D, and it's obvious that they have no way to do this -- no understanding of the scene.

They also could not understand the concept of "back patio" and "side patio" -- relative to the house, when the house was being photographed from different angles. Even when I pointed out which one was the side patio, they would get it confused in a different image.

(I should not be surprised by this, and yeah I know everyone says LLMs have "jagged" capabilities. But to me this shows how bad all the extrapolation of AI I read on say Hacker News has been)

▲

mootothemax an hour ago | parent [-]

I reckon this type of task is crossing capabilities in a way that the LLMs (very quietly, mind) don't support.

An LLM can describe a photo fine, but it cannot tell you the exact coordinates of what it's describing. When I last tested this, you could sometimes get it to lie about the coordinates, but asking it to draw an approximate bounding box around a given feature - total fail, sometimes laughably so.

	▲	post-it 32 minutes ago \| parent [-]
		Yeah, I tried to have AI arrange some furniture-shaped rectangles on a black-and-white floor plan. Impossible. I could painstakingly guide it through writing a floodfill to find the corners of a room and parse out the dimensions listed in that room, but at that point it wasn't using its image-seeing abilities at all, and it still couldn't do things like putting a particular rectangle against the bottom left corner.