Remix.run Logo
▲ simonw an hour ago

Pelicans for 6.2 Sol: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

And for 6.2 Astra: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

Here's an updated comparison grid for the entire GPT-6 family: https://static.simonwillison.net/static/2026/openai-pelicans...

UPDATE: And if you're bored of pelicans, here's what I got for:

> Make a three js animation where yonkers stands up as a big mech (cool stand up motion) and stomps around manhattan and queens and Brooklyn, use openstreetmap mapping data

6.2 Astra (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...

6.2 Sol (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2...

For comparison, here's Fable 5.2 Medium: https://tools.simonwillison.net/yonkers-rises and Gemini 4 Carbon: https://static.simonwillison.net/static/2026/yonkers-mech-st...

▲vopi an hour ago | parent | next [-]

This might be in other pelicans, but 6.2 Sol/Astra is the first time I'm noticing a model shade the leg further in the frame darker to mimic depth (or a shadow).

It's also interesting you can clearly tell 6.2 Astra does a much better job of the above. 6.2 Sol attempts it at higher reasoning levels, but it's crude in comparison.

I wonder if it comes from the (seemingly) recent inclusion of 3D environments in training data.

▲glenjamin 13 minutes ago | parent | prev | next [-]

Interestingly, on the mech thing it seems like only Fable really nailed the brief - the others might look cooler, but Yonkers and the mech aren't directly connected in the same way.

▲boutell 9 minutes ago | parent | prev | next [-]

Fable gets at the mech Yonkers idea in a much more entertaining way than the others.

▲ryanschaefer 17 minutes ago | parent | prev | next [-]

I think this is the first time I’d call the pelican benchmark solved. All levels of effort look good.

▲boutell 13 minutes ago | parent | prev | next [-]

Interesting that pelican quality does not go up monotonically within Astra. Honestly, I think low was the best one.

▲RGS1811 an hour ago | parent | prev | next [-]

The emergence of the beach setting and the scarf is interesting.

▲kzrdude 10 minutes ago | parent [-]

I think a scarf or similar is relatively common with bicycle cartoons, to indicate speed or movement.

▲adverbly an hour ago | parent | prev | next [-]

What's with everyone saying these Pelicans are great?

These are some of the worst regressions that I've seen in a while.

Pay close attention to the stacking layers around the feet, pedals (sometimes there aren't even two pedals), wheel, and two legs. Some pretty obvious mistakes in most if not all of these...

Edit: And what's up with the single arm on the two highest Astra images?

edit2: really? I'm getting downvoted because people don't like that I pointed out a mistake in a Pelican svg? Is that where we're at?

▲preommr 40 minutes ago | parent | next [-]

I have to agree, those results are surprisingly bad and I am not seeing an improvement.

For sol 6.2, some of them are slightly better at higher cost, some of them are way worse and only slightly cheaper (but comparable to terra in quality while being more expensive).

I guess we'll see as more information comes out, but so far not looking good.

▲svachalek an hour ago | parent | prev [-]

The highest Astra images give the pelicans wings instead of arms, which does seem like a valid interpretation.

▲I_Write_It an hour ago | parent | prev | next [-]

They all seem very good, with a lot of attention in detail ! Thx for the Pelicans!

▲BrokenCogs 7 minutes ago | parent | prev | next [-]

No idea what a yonkers is but that fable 5.2 animation blows the gpt 6.2 models out of the water

▲BoldColdHold an hour ago | parent | prev | next [-]

Wow the yellow is leaking into their llms too.

▲tintor an hour ago | parent | prev | next [-]

Sol 6.2 max is still failing on pelican benchmark. Left leg is jammed in chain.

Astra 6.2 max did a similar mistake on the left bicycle pedal.

▲frangonf an hour ago | parent [-]

But Astra 6.2 low and medium didn't make this mistake.

▲globular-toast 41 minutes ago | parent | prev | next [-]

Do we know why they always end with a scarf?

▲phoghed 6 minutes ago | parent | next [-]

They are a classic motif for pilots, birds fly, and there are famous cartoon birds with a scarf, like the ones from The Rescuers or Duck Tales come to mind.

▲pinkyclouds2 34 minutes ago | parent | prev | next [-]

winter is coming

▲ 33 minutes ago | parent | prev [-]
[deleted]
▲firemelt an hour ago | parent | prev | next [-]

any index link for all ur pelicans i want to see fable and opus

▲simonw 25 minutes ago | parent [-]

https://simonwillison.net/tags/pelican-riding-a-bicycle/ has all of them.

▲p11p an hour ago | parent | prev | next [-]

Are you getting any signal out of this anymore? All I can see is "good pelican" and "dogshit pelican". These ones look pretty good.

▲EugeneOZ an hour ago | parent | next [-]

I do. 6.2 Sol xhigh and max show the results comparable to Astra 6 max, and Astra 6.2 only shows any improvement on "max" reasoning effort level.

▲adverbly an hour ago | parent | prev [-]

You're kidding right?

These are some of the worst pelicans I've seen in a while.

Pay attention to the stacking layers near the legs. Sol literally never got it right, and Astra regressed on it multiple times...

Also what is that bending bar from Astra?

▲jay_kyburz 31 minutes ago | parent [-]

When are the Pelicans going to stop being rendered 2d like a child's drawing. We are a million miles away from what a professional artist would come up with.

Update: Here is some Bluey characters riding bikes as a reminder how far these models have to go.

https://www.bluey.tv/characters/uncle-rad/

▲Hacktrick an hour ago | parent | prev | next [-]

I'm not sure if this is a useful benchmark any longer

▲InsideOutSanta 43 minutes ago | parent [-]

They're extremely useful. In fact, I refuse to make infrastructure decisions in a pelican-blind environment.

▲EugeneOZ an hour ago | parent | prev [-]

still the best benchmark! thanks