There’s something about geospatial work that seems to break LLMs. I’ve experienced it a lot with latest versions of both Fable and Opus, and it’s the classic nightmare of confidently-spoken-but-wrong.

Yesterday, I had a relatively trivial bit of work in play using distance-weighting to measure stuff, and Claude did really well / but right at the end, despite having a success metric and lots of great results matching it, Claude insisted my maths was wrong and needed changing.

When i asked why, and pointed out that the suggested replacement would cancel out the distance-weighting entirely, it immediately agreed with me and asked to write its fifth project memory about it.

The other big ones it tends to mess up on without strict guardrails are coordinate systems; and measurements, often converting back and forth between a given nation grid and wgs84 latitude/longitude in ways that lose good amounts of precision; and then custom-coding haversine-equivalents to measure a distance that’s otherwise be a simple subtraction via the national grid coordinates. It’s baffling.

▲

Waterluvian a minute ago | parent | next [-]

Oddly I’ve found it to be shockingly effective lately.

I’ve been renovating a spatial/topology validation system for an OpenLayers driven map editor for robot fleets. I set up a clean interface for spatial indexes and predicates as well as your basic geotoolbox functions.

I was most surprised by a moment where Opus detected that using a union approach to ensure that robot footprints always sat within valid polygons (possibly straddling multiple valid polygons) was slow in testing. So it decided to find the difference between footprint and each polygon within the AABB of it. And if the remaining polygon is zero size, it’s valid. If not, the remaining shape is what is invalid.

I was also surprised just how good it was at recognizing, “in the test data there’s a lot of this kind of feature so we should explore what’s more common: hits or misses?

▲

carodgers 4 minutes ago | parent | prev | next [-]

They're notoriously, hilariously bad at chess, even today. So yes, agree, their shortcomings come much more forcefully and immediately to light when they attempt spatial tasks.

▲

jandrewrogers an hour ago | parent | prev | next [-]

LLMs really don't understand space.

▲

chubot an hour ago | parent [-]

Yesterday I was trying to add a patio to some real estate photos, i.e. "virtual staging".

I uploaded 4 different photos of the same house, from different angles.

I asked Gemini, ChatGPT, and Grok add the same patio to each photo. The results were laughably incoherent (not to mention ugly).

I repeatedly asked them to make the 4 photos consistent in 3D, and it's obvious that they have no way to do this -- no understanding of the scene.

They also could not understand the concept of "back patio" and "side patio" -- relative to the house, when the house was being photographed from different angles. Even when I pointed out which one was the side patio, they would get it confused in a different image.

(I should not be surprised by this, and yeah I know everyone says LLMs have "jagged" capabilities. But to me this shows how bad all the extrapolation of AI I read on say Hacker News has been)

▲

mootothemax an hour ago | parent [-]

I reckon this type of task is crossing capabilities in a way that the LLMs (very quietly, mind) don't support.

An LLM can describe a photo fine, but it cannot tell you the exact coordinates of what it's describing. When I last tested this, you could sometimes get it to lie about the coordinates, but asking it to draw an approximate bounding box around a given feature - total fail, sometimes laughably so.

	▲	post-it 29 minutes ago \| parent [-]
		Yeah, I tried to have AI arrange some furniture-shaped rectangles on a black-and-white floor plan. Impossible. I could painstakingly guide it through writing a floodfill to find the corners of a room and parse out the dimensions listed in that room, but at that point it wasn't using its image-seeing abilities at all, and it still couldn't do things like putting a particular rectangle against the bottom left corner.

▲

nicolamanzini an hour ago | parent | prev | next [-]

I had similar issue with 3d code in threejs. Models used to be quite bad over a year ago. got ok around year ago. the latest are quite good at it.

so i made a simple ab testing eval on fixed prompts: https://threejseval.com

There is a 7 degree o freedom robot arm that is really telling of the improvements in coordinates reasoning:

https://threejseval.com/gallery?eval=robot-arm

▲

tomgp an hour ago | parent | prev | next [-]

This is my experience too, it has got a bit better but nowhere near as quickly as other domains I deal with. I can usually guide the LLM to a decent solution but there's a lot of hand holding and I'm often left wondering if it wouldn't have been quicker for me to just get stuck in and do it the old fashioned way. Maybe because geospatial stuff is pretty niche so is under-represented in the training data relative to yr standard CRUD apps and React front ends which are ten-a-penny?

	▲	an hour ago \| parent [-]
		[deleted]

▲

rfgplk 2 hours ago | parent | prev [-]

Use Astra.

	▲	redhed an hour ago \| parent [-]
		As someone working on a geospatial program, Astra does not seem significantly different to Opus to me. I was surprised myself.