~2 years ago gemini2.5 helped write better kernes for itself and (only) reached 1% efficiency gains. Today we're at 20%.

▲

magicalist an hour ago | parent | next [-]

If you optimize program A and manage to wring out a 1% improvement, and I optimize program B and improve performance by 20%, you can see the problem with trying to infer anything from those two numbers.

Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost. This is a bad comparison.

[1] https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...

	▲	NitpickLawyer an hour ago \| parent [-]
		> This is a bad comparison. How so? First, kernel writing (or ML engineering more broadly) is a highly specialised task. Not everyone can do it. It shows that models are getting better and better at (easily verifiable) hard tasks. And you can "hire" that expertise much easier than you can hire the equivalent meatbags. And more importantly you can "fire" them as soon as the task is done. And then hire them 3 months later, when the new model drops. And so on. Second, 20% gains in inference today gives better end results (i.e. lower overall cost) than 1% in training 2 years ago. Today's models are improving mostly via RL. And RL is highly dependant on fast inference (you want many rollouts for each training scenario). Same for dataset filtering, environment generation, distillation, etc.

▲

dust42 3 hours ago | parent | prev [-]

In 2 years from now we will be at 400%. https://xkcd.com/605/ Also, it is called kernels (you have nitpick in your username)