Regarding the pelican test. Maybe someone will know this.

Modern models can make nearly photorealistic games in threejs. So why do they struggle with these pelicans?

Models seem to have very interesting blind spots. While https://assbench.com/ may seem like a joke, the results are very illuminating. One might expect far better results given the detailed prompt, allowance for render checks and iteration, the number of human figures the models are trained on, and clarity of tool use (threejs).

However, result quality really drops off once you get out of the top 5 or so.

(NSFW note: all buttocks are covered by fairly conservative bikini bottoms and show no genitals, bulges, etc.)

The models that are really good at 3D modeling tend to also be really good at pelicans. I think the skills are strongly related - it's about having good 2D (and now 3D) spatial awareness.

The test is always going to be hard though, because it's an absurd image to ask for.

UPDATE: I like this theory about using an agentic harness that can render and iterate better: https://news.ycombinator.com/item?id=50073879#50074395

"Pelican on bicycle in SVG" is an adversarial example on multiple levels - out of distribution, ill choose of tool for the task, and anatomical impossibility of real pelicans using typical bycicles.

The pelicans are one-shot with not tools: Think a bunch, write a bunch of SVG, done.

The pretty 3d stuff you see are agents controlling a Blender MCP in a loop where they look at the scene, think, adjust the scene, repeat.

A lot depends on what the models were specifically trained for using reinforcement learning. Modern models are generally OK and everything, but they're only really good at the things they have extensive RL for.

> Modern models can make nearly photorealistic games in threejs. So why do they struggle with these pelicans?

Are you sure they're not fetching free asset packs for that?

Have you tried editing SVG by hand?