I do have a private benchmark I use for that. I have a repository with an in-progress codebase and a very large planning document, I tell the new models to simply finish everything in the planning document, and I never push those changes. All of the recent models have essentially gotten it perfect which means that the models are all equivalently good for my level of needs.

I've mostly been running LLMs on my own hardware so the phrase "been at it for about 20 minutes" gave me pause. My first instinct was to ask "but on what hardware?" but I suppose one advantage of closed models is that there's a somewhat-consistent cognitive-effort-to-time ratio.

Alternatively Mitchell and Webb https://www.youtube.com/watch?v=_pDTiFkXgEE

Just today I asked Claude Fable to take an SVG map and add a circle with a 150 mile radius around a specific city, cutting off the circle at the edge of certain boundaries. It came back a minute later with more or less exactly what I wanted, saving me maybe 5 to 10 minutes of photoshop time. It’s almost certainly not perfect, but I didn’t need it to be perfect, just a legible representation for a group of 20ish people.

I used Claude to plan some landscaping, and it gave me SVG diagrams of flower beds showing where to place plants and their approximate mature size. It was pretty useful.

Great for charts, illustrations etc, can do much more with SVG than pretty pictures. Good spatial understanding in an SVG can be really nice for model to have

I used it to generate some icons for my app recently, I found it a pleasantly useful feature already in its current state. Before this I would’ve had to close the code editor, open something like Illustrator, use something like a tablet (or struggle with the mouse and keyboard), and export the result. For something like a generic icon for a button, or a simple geometric figure, this always felt too tedious. On the other hand, my apps looked visually impressive before the AI and people valued my artistic ability more. So there’s that… Not every software engineer also had an eye for design. With AI generation, they can bridge that gap.

AI can generate a fairly satisfactory SVG for a favicon now (programmer art quality at least).

Even if the models don't break through any particular "uglier than sin" barrier, with a bit more work, presumably the SVGs could become importable into an editor that would let a human apply taste and discretion. Seems to me like a heck of a head-start.

As for conventional diffusion-model stuff, I happen to think there are some pieces of AI art that still look really good even knowing they're AI.

Would it be useful if the models were actually good at it?

Vibecoding a SVG based metroidvania as we speak! This is gonna be lit!

I use LLMs for 3D CAD design in OpenSCAD.

There seems to be a very strong correlation between models that are good at SVG and models that are good at 3D CAD.

Anecdote I know, but there does seem to be generalization going on here.

If a model can improve at drawing "X doing Y" and that prompt wasn't in the training set then it means it has improved its internal mapping from text-to-spatial-to-text.

No, the point is that a general-ish ability to draw good SVGs is a useful ability in itself. People need SVGs for all sorts of purposes, and if AI can generate one for them, that's mostly useful (discussions about art and employment etc notwithstanding).

That said, I think this would correlate relatively little with general programming ability. They're not unrelated, of course, but being able to generate code that paints an accurate + esthetically pleasing image is quite different from generating code that achieves a non-spatial goal.

I don't see why that's true. LLMs don't have to only be good at code-writing.