| ▲ | The Pelican comparison grid for Astra is pretty interesting(simonwillison.net) | |||||||||||||||||||||||||||||||
| 25 points by csmantle 3 days ago | 7 comments | ||||||||||||||||||||||||||||||||
| ▲ | suprjami 3 days ago | parent | next [-] | |||||||||||||||||||||||||||||||
> The Astra pelicans are much better. It's absolutely in the training data now. Time to retire this (previously mildly amusing) benchmark as frontier models are now pelicanmaxxed. | ||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||
| ▲ | williamDafoe 2 days ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||
1. Head tube should aligned with the fork. GPT-5.6 Luna is particularly terrible here. 2. Bikes are almost always depicted going rightwards so you can see the drivetrain. 4 failures in this. 3. Only ~7 of the 23 models achieved a diamond bicycle frame shape 4. Almost all the models have the bird directly over the crankset; as little as 4 or 5 have the birds sitting on a set-back bicycle seat. | ||||||||||||||||||||||||||||||||
| ▲ | amluto 2 days ago | parent | prev [-] | |||||||||||||||||||||||||||||||
Is it just me or is it kind of remarkable that all the remotely competent pelicans on bicycles kind of look the same? I assume that these have been in the training set for quite a while, but that they’re still a bit hard for the models because it’s not covered by the RL pipelines. | ||||||||||||||||||||||||||||||||