But theoretically, what would prevent the labs from RL'ing the models on "weight-space programming" tasks so that they get better at them? Perhaps we're just at the beginning, like how coding agents sucked 2 years ago.

It still seems like a promising approach for RSI, distillation, fine-tuning, and a whole host of other tasks.

▲

_ache_ 2 hours ago | parent | next [-]

The context.

It's impossible to train on raw document > 1MB (I don't know the exact limit, but there is one). You can't vectorize weights (or can you?!) and the output context is limited. So you can't train a big model on it's previous weight or others models weights.

The output context can be "extended" using agents but I doubt anything coherent may be outputted using that technique.

	▲	danielmarkbruce 37 minutes ago \| parent [-]
		Of course this is the problem. It's amazing that for all the commentary around AI models the number of people who really understand even the basics approaches zero.

▲

jfyi 2 hours ago | parent | prev | next [-]

Yeah, this was my thought (as a layman) as well.

It seems some level of interpretability, even if it requires tooling, could result in the ability to spin out synthetic training data sets from other models in an automated fashion.

▲

ex-aws-dude 3 hours ago | parent | prev [-]

They might even discover some way to program the weights proportional to an error signal

Now that would be crazy

	▲	vkgcr 3 hours ago \| parent [-]
		I know you're being sarcastic, but I meant prompting an LLM with "generate me a X billion parameter model for doing Y task" and it would just output the weights. Now THAT would be crazy