Gemini 3.8 flash seems to (subjectively) be the outlier in terms of performance to cost ratio?

pixelesque 5 hours ago | parent | next [-]

Its giraffe / grandfather clock one is pretty bad... (two necks? wearing a suit?)

Weird as well, it's clearly pulled out some 1884 patent on clock designs, and a quick ddg/google doesn't show it as anything to do with grandfather clocks.

▲

simonw 2 hours ago | parent | prev | next [-]

Google Gemini are the only team who have openly had staff deliberately spend time on SVG performance: https://twitter.com/sunjiao123sun_/status/202455551655137292...

See also this Jeff Dean tweet showing off their animals-in-vehicles abilities: https://twitter.com/JeffDean/status/2024525132266688757

▲

gpt5 2 hours ago | parent [-]

It’s very likely they all add svg generation into the training data. It’s part of the reason it’s no longer a good benchmark (unless you need to generate SVGs).

	▲	simonw an hour ago \| parent [-]
		It's about more than just training data. Every model since GPT-2 had SVGs in the training data, because they all used a scrape of the Web and the Web is full of SVGs. What's unique about Gemini is that they actively worked to get better results for their SVGs - maybe RLHF, maybe RLVF of some sort.

▲

GaggiX 6 hours ago | parent | prev [-]

Gemini 3.8 Flash results are often not very coherent but it does put a lot of shading and details to hide the fact.