| ▲ | simonw an hour ago | ||||||||||||||||||||||
Pelicans for 6.2 Sol: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... And for 6.2 Astra: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... Here's an updated comparison grid for the entire GPT-6 family: https://static.simonwillison.net/static/2026/openai-pelicans... UPDATE: And if you're bored of pelicans, here's what I got for: > Make a three js animation where yonkers stands up as a big mech (cool stand up motion) and stomps around manhattan and queens and Brooklyn, use openstreetmap mapping data 6.2 Astra (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2... 6.2 Sol (Ultra in Codex): https://static.simonwillison.net/static/2026/yonkers-gpt-6.2... For comparison, here's Fable 5.2 Medium: https://tools.simonwillison.net/yonkers-rises and Gemini 4 Carbon: https://static.simonwillison.net/static/2026/yonkers-mech-st... | |||||||||||||||||||||||
| ▲ | vopi an hour ago | parent | next [-] | ||||||||||||||||||||||
This might be in other pelicans, but 6.2 Sol/Astra is the first time I'm noticing a model shade the leg further in the frame darker to mimic depth (or a shadow). It's also interesting you can clearly tell 6.2 Astra does a much better job of the above. 6.2 Sol attempts it at higher reasoning levels, but it's crude in comparison. I wonder if it comes from the (seemingly) recent inclusion of 3D environments in training data. | |||||||||||||||||||||||
| ▲ | glenjamin 13 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
Interestingly, on the mech thing it seems like only Fable really nailed the brief - the others might look cooler, but Yonkers and the mech aren't directly connected in the same way. | |||||||||||||||||||||||
| ▲ | boutell 9 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
Fable gets at the mech Yonkers idea in a much more entertaining way than the others. | |||||||||||||||||||||||
| ▲ | ryanschaefer 17 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
I think this is the first time I’d call the pelican benchmark solved. All levels of effort look good. | |||||||||||||||||||||||
| ▲ | boutell 13 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
Interesting that pelican quality does not go up monotonically within Astra. Honestly, I think low was the best one. | |||||||||||||||||||||||
| ▲ | RGS1811 an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
The emergence of the beach setting and the scarf is interesting. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | adverbly an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
What's with everyone saying these Pelicans are great? These are some of the worst regressions that I've seen in a while. Pay close attention to the stacking layers around the feet, pedals (sometimes there aren't even two pedals), wheel, and two legs. Some pretty obvious mistakes in most if not all of these... Edit: And what's up with the single arm on the two highest Astra images? edit2: really? I'm getting downvoted because people don't like that I pointed out a mistake in a Pelican svg? Is that where we're at? | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | I_Write_It an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
They all seem very good, with a lot of attention in detail ! Thx for the Pelicans! | |||||||||||||||||||||||
| ▲ | BrokenCogs 7 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
No idea what a yonkers is but that fable 5.2 animation blows the gpt 6.2 models out of the water | |||||||||||||||||||||||
| ▲ | BoldColdHold an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
Wow the yellow is leaking into their llms too. | |||||||||||||||||||||||
| ▲ | tintor an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
Sol 6.2 max is still failing on pelican benchmark. Left leg is jammed in chain. Astra 6.2 max did a similar mistake on the left bicycle pedal. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | globular-toast 41 minutes ago | parent | prev | next [-] | ||||||||||||||||||||||
Do we know why they always end with a scarf? | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | firemelt an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
any index link for all ur pelicans i want to see fable and opus | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | p11p an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
Are you getting any signal out of this anymore? All I can see is "good pelican" and "dogshit pelican". These ones look pretty good. | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | Hacktrick an hour ago | parent | prev | next [-] | ||||||||||||||||||||||
I'm not sure if this is a useful benchmark any longer | |||||||||||||||||||||||
| |||||||||||||||||||||||
| ▲ | EugeneOZ an hour ago | parent | prev [-] | ||||||||||||||||||||||
still the best benchmark! thanks | |||||||||||||||||||||||