I'm glad someone ran the numbers on this. Every single Simon Willison post of an SVG is followed with someone dismissing it saying "I'm sure they train on it by now." This is despite a good blog post with sound logic on how easy that is to catch. [1] Glad to see someone took the time for a quantitative analysis of dumb little animals riding dumb little bikes.

1. https://simonwillison.net/2025/Nov/13/training-for-pelicans-...

▲

unholiness 9 hours ago | parent | next [-]

I don't think this small amount generalization to other animals and vehicles is strong evidence they haven't trained on this, either directly or more generally.

▲

conception 7 hours ago | parent [-]

Honest question how could they possibly train on this as there are no good SVG pelicans to train off right? So they’re just training off a bunch of bad ones which should lead to just bad pelicans, but the pelicans are getting better.

	▲	unholiness 5 hours ago \| parent \| next [-]
		Training on generating SVGs directly at all is already fairly niche. Generating full scenes with a cartoony character is even nicher. But there's plenty of non-pelican cartoony SVG content out there (created, not written, by humans with vector design tools), and more importantly, plenty of vision models to give feedback on the output (just raster as a png). You could easily hill climb this niche skill, if you cared.
	▲	alexthehurst 7 hours ago \| parent \| prev \| next [-]
		It’s not hard for a visual model to score the quality of that output though, which would be a pretty good fitness function.
	▲	raincole 4 hours ago \| parent \| prev [-]
		They can easily afford 1000 human made pelican svg files if they want. I think you underestimate how much resources SotA AI companies have. (I'm not saying that they did that. I'm just saying they can.)

▲

elicash 8 hours ago | parent | prev [-]

There's that version of the argument version, but there's also the softer version: that there used to be no training material of illustrated pelicans on bicycles, but now you have actual artistically talented individuals drawing it and that could improve the performance even though the AI labs are sucking it up no differently than everything else.

This post proves that hasn't happened yet, either. Although maybe the bad results posted online are being trained on and that explains the UNDER performance.