| ▲ | simonw 3 hours ago |
| Six effort levels (minimal, low, medium, high, xhigh, max), so here are six pelicans (actually five, "high" didn't return a complete SVG): https://tools.simonwillison.net/markdown-svg-renderer?url=ht... There's usually a reasonably strong correlation between thinking level and visual quality of the pelican. This is the first time I've seen that correlation appear to break - more reasoning tokens doesn't seem to produce an obviously better SVG. |
|
| ▲ | ezst 2 hours ago | parent | next [-] |
| Wouldn't that be a sign that those models were trained to optimise that test case specifically, as in, by exhibiting equal or higher performance while using less power? |
|
| ▲ | meric_ 3 hours ago | parent | prev | next [-] |
| Wow, minimal seems to be the best here interestingly. Only 10 reasoning tokens so basically none? |
|
| ▲ | eterm 3 hours ago | parent | prev | next [-] |
| Did minimal just hand it off to something else? It seems completely out of line with the rest. |
|
| ▲ | imimayj1337 3 hours ago | parent | prev | next [-] |
| It does seem pretty insane to me that only on max setting (and after waiting 27 seconds) did you actually manage to receive an SVG where it appears that the pelican's legs are straddling the bike correctly. Edited to make clear what it looks like to me and what I was looking for. |
| |
| ▲ | swatcoder 3 hours ago | parent | next [-] | | They're not. It's an illusion from the composition. If you look at the code for each of these (and probably most across models), you can see that the drawing elements are organized by concept. Presumably, this helps accuracy by keeping attention on one concept at a time, but you end up with a stack of conceptualized objects and lose the ability to interleave things. No doubt these could all do better with a more thorough and specific prompt that coached them away from that technique, but otherwise it's a very subtle detail that a "thinking" process needs to stumble upon independently. | | |
| ▲ | flir 2 hours ago | parent [-] | | You could use a second prompt to z-order the objects (I tend to break down tasks and pipeline them, rather than trying to stuff everything in a one-shot prompt. Seems to work better for me, could be a holdover from the early days). |
| |
| ▲ | latexr 3 hours ago | parent | prev [-] | | They’re not. Note how both pedals are on the same side of the frame. That’s wrong! |
|
|
| ▲ | permalac 3 hours ago | parent | prev | next [-] |
| Is there a problem if I like the minimal more than the others? |
| |
|
| ▲ | AlecSchueler 2 hours ago | parent | prev | next [-] |
| Any idea why there's so much convergence on the fish in a basket idea? |
| |
| ▲ | swatcoder 2 hours ago | parent [-] | | Same reason they're all facing right with the same composition, they all have three motion lines, they all have a yellow sun and 2-3 blobby clouds, they're all on two-lane pavement with a white dashed lane separator. And the same reason Claude has all its notorious Claudisms in prose. The shorter and vaguer a prompt you give to an LLM, the more it fills in further details using associations they've developed through training. These are ley lines that the models gravitate towards. Diffusion models too. It's why lazy AI use for creative work always comes across as lazy in the wild and always will. When you use AI for "creative" purposes yourself, it feels like magic to see a pelican riding a bicycle at all, and you're impressed by all those cool details it added. But then 10, or 100, or 10M people throw more or less the same trivial prompts at it and we start to recognize all those "ley line" associations fill in the gaps the same way again and again and again. | | |
| ▲ | viraptor 35 minutes ago | parent | next [-] | | To be fair, people have a genuine preference for images with things facing to the right. And lots of drawings will do 3 motion lines. You'd likely draw a dashed rather than solid separator too, or it would look like a random line. Lots of the AI habits are there because of choices we make. | |
| ▲ | mvkel an hour ago | parent | prev [-] | | This is interesting. There's a balance between: - simple prompt for complex output (bad) - complex prompt for simple output (bad) - simple prompt for simple output (good) - complex prompt for complex output (.... is this good or bad?) |
|
|
|
| ▲ | mgdev an hour ago | parent | prev | next [-] |
| Fidelity. |
|
| ▲ | phyrex 2 hours ago | parent | prev | next [-] |
| I like that from xhigh on up its wearing a helmet. |
| |
|
| ▲ | sjacob 3 hours ago | parent | prev | next [-] |
| Incredible. How do you get these out so fast? |
|
| ▲ | jolt42 3 hours ago | parent | prev | next [-] |
| Pelican is the new teapot |
|
| ▲ | djmips 2 hours ago | parent | prev | next [-] |
| birds at minimal and return at max |
|
| ▲ | small_model 2 hours ago | parent | prev | next [-] |
| Not sure this is a good way to benchmark LLM's, no offence, but a single SVG is hardly a measure of a models capability. |
|
| ▲ | AmazingTurtle 3 hours ago | parent | prev [-] |
| [dead] |