Is there any point anymore regarding this svg test? I would not be surprised if in the training they're fine tuned for this task too

▲

tintor 2 hours ago | parent | next [-]

It would be very embarrassing for any lab to benchmaxx the pelican on bicycle svg prompt, since it would be very easy to detect it by varying the prompt.

	▲	nojs 44 minutes ago \| parent [-]
		The amount of discussion around it means that the test and all the reviews of results, images, approaches etc are implicitly included in training data. It’s not deliberate “benchmaxxing” but things that are discussed a lot online are naturally things that LLMs learn better.

▲

BeetleB 2 hours ago | parent | prev [-]

You win this thread's prize:

https://news.ycombinator.com/item?id=49538333