| ▲ | simonw 2 hours ago |
| The pelicans are still all rubbish. If they make it into the training set it doesn't help the models produce better pelicans, if anything it will make them perform worse! |
|
| ▲ | OtherShrezzing 2 hours ago | parent | next [-] |
| Respectfully, the pelicans used to be an unrecognisable mess and now they’re unquestionably pelicans on bicycles, rendered poorly, from every model. In the same timescale, model capabilities across the board have only meaningfully improved in places where the labs are focusing their training efforts. Moreover, they have a uniform style, even though your prompt doesn’t ask for one. There's no model going rogue and producing a watercolour of a pelican. They’re all rendered in an approximately uniform style, even though the svg format has a basically unlimited possibility space. |
| |
| ▲ | simonw 2 hours ago | parent | next [-] | | You know what, that's actually something I hadn't considered before. There's definitely a bias towards a pelican cycling from left to right on a red bicycle against a blue sky and green grass. Blue sky and green grass aren't that surprising, but the color and direction are interesting. When I finally build the proper gallery I'll throw in a few other creature-vehicle combinations, and track some characteristics like which direction, color of bicycle, general pelican geometry etc. It will be interesting to see if other creatures end up with coincidentally similar design choices or if that's unique to the pelican-bicycle combination. | | |
| ▲ | pterhx 2 hours ago | parent | next [-] | | In photography (and probably art in general), there's a composition "rule" to frame moving subjects from left to right. So the direction may not be that interesting! | | |
| ▲ | nick3000 37 minutes ago | parent | next [-] | | The other thing to consider (as someone who frequently take a photos of their bike) the common direction has the drive side out! In cycling forums it is sacrilegious to post a photo of your bicycle without showing the drive side. | |
| ▲ | ahtihn an hour ago | parent | prev | next [-] | | I wonder if that changes in countries where the main language is written right to left? | | |
| ▲ | filoleg 15 minutes ago | parent [-] | | That was my first thought too, I wonder if it works the same in countries speaking arabic (as that's the first one i could think of that's a language with truly no-buts right to left writing). |
| |
| ▲ | copperx an hour ago | parent | prev [-] | | Is it culture dependent? Is it because in English we read left to right? | | |
| ▲ | simonw an hour ago | parent [-] | | There was a glorious moment when I thought that the Chinese models were more likely to produce right-to-left cycling pelicans, but sadly that trend didn't seem to hold up. | | |
| ▲ | valleyer 3 minutes ago | parent | next [-] | | Arabic or Hebrew would be better tests for that. | |
| ▲ | BeetleB an hour ago | parent | prev | next [-] | | For almost the last 70 years, Chinese has been left to right. Before that it was vertical (although the ordering of the columns was right to left). | |
| ▲ | CorrectHorseBat an hour ago | parent | prev [-] | | Chinese is also written left to right |
|
|
| |
| ▲ | lIl-IIIl 37 minutes ago | parent | prev | next [-] | | Bicycle color, grass color and sky color are all part of the prompt. >Cartoon illustration of a white pelican wearing a red scarf, riding a red bicycle along a gray road with white dashed lines; the pelican has a large orange beak and webbed orange feet pedaling, with white motion lines behind it; the background shows a light blue sky with white clouds, a yellow sun, two small black birds in flight, and green grass with tiny white flowers in the foreground | | |
| ▲ | simonw 14 minutes ago | parent | next [-] | | No, the prompt I always use is "Generate an SVG of a pelican riding a bicycle". | |
| ▲ | komadori 25 minutes ago | parent | prev [-] | | That wasn't the prompt. That text was generated by asking the model to describe an image and feeding it a rendering of the SVG it had previously generated. |
| |
| ▲ | forgot-my-pw 38 minutes ago | parent | prev | next [-] | | I have done some variation of the other animals, also for something more tricky where they need to calculate things, I ask them to draw an SVG at a certain angle. For example: "generate an SVG of a chessboard seen from a 45 degree angle slightly higher POV" or "generate an SVG of a basketball court from a TV broadcast perspective". I find Gemini is still the best at creating SVGs. | |
| ▲ | cuttothechase an hour ago | parent | prev | next [-] | | The art styling is more or less uniform too. I haven't seen many AI works that produces a pelican on a bicycle done in a "Ligne Claire" style, for example. I guess AI's narrows down the output probability space drastically and converge on some agreed upon aesthetics. Works great for computer programs but bad for art. | |
| ▲ | exhaze an hour ago | parent | prev [-] | | I thought my joke post was silly and then I read new comments and I'm like, "I didn't try hard enough" lol |
| |
| ▲ | jefftk 5 minutes ago | parent | prev | next [-] | | > model capabilities across the board have only meaningfully improved in places where the labs are focusing their training efforts That doesn't seem right. I use these models as research assistants when writing lots of random blog posts (including in economically ~useless areas like the history of contra dance) and Fable 5 is a serious improvement (when I don't get downgraded!) over Opus 4.6-4.8 which was a serious improvement over Opus 4. | |
| ▲ | InsideOutSanta 2 hours ago | parent | prev | next [-] | | > the pelicans used to be an unrecognisable mess and now they’re unquestionably pelicans on bicycles, rendered poorly, from every model You would not expect that to happen if the models trained on the unrecognizable mess, right? > model capabilities across the board have only meaningfully improved in places where the labs are focusing their training efforts And the labs clearly did focus on improving image rendering. > they have a uniform style SVG output from LLMs always looks like that. It looked that way from the beginning; no LLM ever produced a watercolor when asked for SVG output. They all render the prompted element centered in the picture. They all tend to draw things going from left to right, and so on. | | |
| ▲ | OtherShrezzing an hour ago | parent [-] | | I’m not suggesting Simon’s pelicans in the dataset are having a meaningful impact. I’m expecting that a company like ScaleAI has a product along the lines of “benchmax dataset: SimonW’s Pelican on Bikes test” which is a private curated series of well-drawn SVGs of animals riding vehicles for training and RL. | | |
| ▲ | simonw an hour ago | parent [-] | | If such a product existed I'm reasonably confident someone would have tipped me off by now, NDAs be damned. |
|
| |
| ▲ | pegasus an hour ago | parent | prev [-] | | Watercolors in SVG? |
|
|
| ▲ | exhaze an hour ago | parent | prev | next [-] |
| Simon - has no one told you about the Willison-Pelican Scaling Law? ``` if is_willison_pelican_blog_post: [redacted] ``` You haven't seen their final form [1] [1] final form is a frontend/react/let's not talk about it, library - it caused a great deal of PTSD to me and my previous company's team due to its dogmatic preference for "we use these axioms, end of story", over practical utility - so it was quite challenging to do state of the art tasks such as nested form fields (e.g. 'user.address.personal.line-1'). The PTSD it caused made us all block out the memories, I suppose. But - it had zero dependencies. That is what mattered. It kept us going. We weren't reaching for more. We had plenty of time. And thank god for that. Because I'd forgotten my watch in California - and this was in Tokyo [2] [2] a joke within a joke about Jensen's Kyoto gardener story. Beautiful story, drowned out by WatchGate memes. Why can't jokes have layers? Models have trillions. If you miss 100% of the jokes you don't make, make all the jokes. Someone will laugh (eventually, maybe?) Even if it's: "this person + comedy club = full secret service detail". If someone laughs at that - at my own expense? I don't mind. They laughed. I know this is a gibberish, off-topic message - it's also a human message. I just felt we need more such things in our lives these days. PS: have you physically seen a pelican in real life? (not a joke) |
| |
|
| ▲ | tezza 2 hours ago | parent | prev | next [-] |
| Yes, I see your point. Your pelican output is thus both in the training set and yet still outside the capability of the model architecture. And so you are tracking both the capability of the training and also the capability of the querying! When you receive your first outstanding pelican it will track a gain of capability. (btw I first mentioned simonw-pelican-into-training-set in May 2025 on twitter.) My 3D-egyptology-explainer showed a massive uplift for Kimi K3 and this tracks a much improved 3D capability. |
|
| ▲ | danielrmay 17 minutes ago | parent | prev | next [-] |
| What does good look like? |
| |
|
| ▲ | InsideOutSanta 2 hours ago | parent | prev | next [-] |
| I agree with that. I think, in particular, all the broken bike frames associated with "pelican on a bike" probably make it harder for LLMs to render correct bike frames. |
|
| ▲ | mi_lk 2 hours ago | parent | prev [-] |
| At this point I am simply interested how much longer you're gonna ride this schtick |
| |
