> If it's on HN and is a meme at this point, it will end up in the training set.

> It's kind of fun to imagine that there is an intern in every AI company furiously trying to get nice looking svg pelicans on bicycles.

Google listened and assigned a few interns, I guess, for the task

▲

JoshTriplett 6 hours ago | parent | next [-]

That's exactly what I was thinking when I saw https://www.assbench.com/ posted a while back: "oh no, this is going to be somebody's job".

▲

codethief 2 hours ago | parent | next [-]

Unfortunately, for me the renderings crash or hang in both Firefox for Android and Chromium/Vanadium on a Pixel 10 Pro. :\

▲

ngruhn 6 hours ago | parent | prev | next [-]

omg that's hilarious

▲

NoDodgeQuestion 5 hours ago | parent | prev | next [-]

prompt is to detailed (llm written?) better test of model creativity is to not constrain prompt

▲

rubyn00bie 5 hours ago | parent | prev [-]

NGL, I think this might be my favorite benchmark, ha. While the Chinese labs asses are a bit off, I certainly have big hopes for their next versions. Their hearts are certainly in the right place.

	▲	gedy 5 hours ago \| parent [-]
		It's the smaller training data /s

▲

eli 7 hours ago | parent | prev | next [-]

SVG is useful for actual tasks, not just looking good in a silly "benchmark"

▲

CuriouslyC 7 hours ago | parent | next [-]

Beyond SVG, these things have been RL'd on 3d reconstruction (which carries over), and I expect given the public reaction to recent frontier advancements in 3d capabilities that this will get allocated a larger share of the post training budget in the future.

▲

PorciiVorbesc 7 hours ago | parent | prev | next [-]

What I'd kill for would be an app where you take a pic of scribbles and drawings on a whiteboard/notebook and the app turns it into SVGs you can then continue to manipulate digitally on PC/tablet.

▲

gh0stcat 7 hours ago | parent | next [-]

Why would you kill for this? Why not just generate svgs with any editing software?

	▲	PorciiVorbesc 7 hours ago \| parent [-]
		>Why would you kill for this? Figure of speech. >Why not just generate svgs with any editing software? How do you do that from the whiteboard/notebook?

▲

xnx 6 hours ago | parent | prev | next [-]

Inkscape and https://github.com/visioncortex/vtracer can both do this.

	▲	matsemann 5 hours ago \| parent [-]
		They can represent something in vector, but it's always very hard to use the result for anything else. Like, the paths doesn't make sense, aren't grouped to the logic elements etc.

▲

eli 7 hours ago | parent | prev [-]

ChatGPT does that decently well

▲

PorciiVorbesc 7 hours ago | parent [-]

Last time I tried, it did not. That's why there's a place on the market for an app just tuned to handle the transition seamlessly, rather than having you fucking around with prompts in LLMs.

▲

TeMPOraL 5 hours ago | parent [-]

I for one don't want a whole app for that, not anymore. It's a small task whose result is an input to something else - ideal for a tool for LLM (or script) to use.

▲

PorciiVorbesc 4 hours ago | parent [-]

>I for one don't want a whole app for that

Everyone says that until ms, apple, google, etc introduces this polished feature as part of their tablet imaging editing/document collaboration office suite. Then everyone can;t get enough of it.

> It's a small task whose result is an input to something else - ideal for a tool for LLM (or script)

Have you actually tried this? If it's a small task then why can't LLMs actually do it? Kind of like the pelican test. Every LLM can do a SVG pelican but most of the results are unusable in practice for production/productivity purposes.

Every LLm can extract some half-broken SVG hallucination of your jots, but I'm asking for an app that's polished and works accurately 99% of the time, which means a dedicated app/model tuned specifically for this task, and nobody makes this yet.

	▲	TeMPOraL 2 hours ago \| parent [-]
		> Everyone says that until ms, apple, google, etc introduces this polished feature as part of their tablet imaging editing/document collaboration office suite. Then everyone can;t get enough of it. I don't feel like I'm a representative sample of general userbase, so I won't dispute this too hard. For me, it's rare but it does happen. Still, 90% of my mobile experience is "how is this so broken, why is it missing the obvious follow-up feature that completes a workflow, and why, again, am I not using a laptop for this?". > If it's a small task then why can't LLMs actually do it? What if they actually can? This is exactly what Karpathy was talking about the other day[0]. If your only LLM experience is the default chat app in chat mode, then this doesn't make sense. If your experience includes using agentic harness like Claude Code/Cowork / Codex/ChatGPT Work, then it's obvious. The LLM app in "chat mode" will just zero-shot you a pelican, or digitize your notes into SVG, and maybe improve on it a bit if you complain about the result. The same app[1] in "agentic mode" will digitize your notes, refine the SVG against the original image, reduce it to probable strokes based on high-frequency patterns in the photo and knowledge of how people write, OCR it, save both the text and SVG to your library, then extract parts you wanted and insert them into a document, typeset the rest of it, and print it on your printer - all in response to a single prompt. LLMs in agentic mode are very good at stringing long and advanced workflows; today, if you give them shell access or computer use, they will use regular apps for you if that's the easiest way to accomplish the goal. > Have you actually tried this? Yes![2]. That's why I can confidently say: there is no limit to this pattern. LLMs can handle apps just fine; giving them tools just saves time and tokens. Another example: I'm currently resurrecting a 30 year old game, and the other day I asked Claude to hack on a way to run it headlessly, so I can use my computer while it plays with it. It did, and then immediately leveraged it to just causally run several instances of the game in parallel, entirely in the background, whenever it needed to playtest something. -- [0] - https://news.ycombinator.com/item?id=50047422 - was on the frontpage two days ago. [1] - Claude and ChatGPT now have both modes in their mobile, web and desktop apps. I personally use Claude Code a lot, but exclusively through these surfaces (I can't be arsed to deal with TUI at home anymore). IMO, these tools are already within reach of regular users - the users just didn't notice it yet. [2] - Not with SVGs, but equivalent with voice recordings, and yes, Claude Code set up and is managing the whole local processing pipeline. Recordings go in, Markdown (or more advanced signal processing) comes out, and in itself, it's not a task, but merely a step.

▲

slyrus 5 hours ago | parent | prev [-]

Of course it would be nice if google actually added SVG support to google slides...

▲

pwython 7 hours ago | parent | prev | next [-]

We all said it: https://news.ycombinator.com/item?id=46978726 (8 months ago)

> How many pelican riding bicycle SVGs were there before this test existed? What if the training data is being polluted with all these wonky results...

But Simon has demonstrated before that asking for any animal doing something silly will still get similar results from the same model.

▲

simonw 4 hours ago | parent [-]

I recently learned that there is an animated pelican riding a bicycle in 1988's Who Framed Roger Rabbit! https://simonwillison.net/2026/Sep/18/the-creative-spirit-of...

▲

thinkingemote 4 hours ago | parent [-]

Do you think this might be a kind of subconscious origin of your idea? A second of a move you saw in your early life / teens stayed with you as an example of an absurd thing to draw?

	▲	simonw an hour ago \| parent [-]
		It's possible, but I think it's more likely that I was inspired by the TikZ unicorn thing, my love of pelicans (we have thousands of them in the harbor ten minutes from my house), and my knowledge that bicycles are difficult for humans to draw.

▲

orbital-decay 7 hours ago | parent | prev | next [-]

It's still clearly not a part of any model's training set in the sense of pelicanmaxxing in particular. It's either a capability to generate arbitrary SVGs (or even something broader), or no specific training at all.

▲

nomel 5 hours ago | parent [-]

Yet, they're all almost exactly the same in style and composition.

▲

orbital-decay 5 hours ago | parent [-]

Certainly not because all labs are pelicanmaxxing on the same exact image. Most patterns are similar everywhere. That's the result of training on the same datasets and scraped web contaminated with other models' outputs, combined with similar training methods and resulting mode collapse. Try if e.g. dolphins knitting socks in spacesuits are similar on all/most models.

	▲	nomel 12 minutes ago \| parent [-]
		This and your original comment aren't compatible.

▲

afavour 7 hours ago | parent | prev | next [-]

Certainly true but I think the question is “does targeting the SVG pelican mean that the quality of all SVG output improves?” If so it’s a worthwhile thing to target.

▲

compiler-guy 7 hours ago | parent [-]

It's a little bit like any benchmark, especially microbenchmarks. It is useful as far as it reflects the broader use cases, but the risk of overfitting to the specific task and not the broader ones is very high.

	▲	DoctorOW 6 hours ago \| parent [-]
		https://xkcd.com/2899/

▲

7 hours ago | parent | prev | next [-]

[deleted]

▲

mlmonkey 6 hours ago | parent | prev | next [-]

Sadly, this is all that managers know to do: take a metric and try to optimize the heck out of it. They don't care about research, technology, "don't be evil", etc. And Google is run by managers now.

Expect Google models to be benchmaxxed as much as possible, but don't expect commensurate performance on real-world tasks.

▲

anthonyrstevens 6 hours ago | parent | next [-]

>> this is all that managers know to do: take a metric and try to optimize the heck out of it. They don't care about research, technology

This is not true. It is needless negative hyperbole that IMHO doesn't belong on HN.

	▲	mlmonkey 5 hours ago \| parent [-]
		I'm speaking from 30 years experience in the Valley, having worked at several big companies.

▲

agar 5 hours ago | parent | prev [-]

While this might be overly cynical, it's also a powerful validation of the exact feedback loop that you want to have in an organization.

If what you say is true, then the next set of benchmarks will be written to illuminate the true reflection of real-world tasks. Then those get benchmaxxed so managers can show "numbers go up."

Repeat enough times, and you get materially and genuinely improved products.

The burden is on the press, outside researchers, and general tech industry to both write and care about the metrics that matter.

▲

kalcode 5 hours ago | parent | prev | next [-]

For sure, they saw your comment and got some interns on it.

▲

dominotw 6 hours ago | parent | prev | next [-]

They said they are training for svg not pelican svg specifically

▲

gcanyon 7 hours ago | parent | prev | next [-]

Maybe so, and yet it's still not enough -- the pelicans all have flaws.

They look remarkably good, sure -- but not perfect.

▲

sulam 7 hours ago | parent | prev [-]

I end up getting SVG output for all kinds of things, and never once have I asked for, or gotten, a pelican.