My somewhat uninformed guess is that the weights of the target network (Qwen or Gemma), when they reach the context of the larger editor model (Astra), the attention process is doing some sort of PCA on the weights, conditional on the prompt ("make this better at games"), fetching the relevant weights from its own network (which serves as the larger database), transforming it into whatever latent space the target model is amenable to and pasting them. If this is the right intuition, it is amazing what attention is able to achieve. As a test, I think smaller networks won't be able to make big ones better using the same process.

EDIT: I think the bigger implication is that backprop is not the only game in town.

▲

adw 5 hours ago | parent | next [-]

https://xingjianbai.com/reactive-optimizers/

from this month

▲

GolDDranks 5 hours ago | parent | prev | next [-]

> fetching the relevant weights from its own network

They don't have usually access to their own weights?

▲

theowaway213456 5 hours ago | parent [-]

I think they're saying this is all happening in latent space through pure reasoning, and "fetching" is just a metaphor for that, not a literal tool call. I do believe it's possible for models to learn the capability to "introspect" their own weights during training.

	▲	charm137 5 hours ago \| parent [-]
		Yes, I should have been more precise - I meant the information embedded in their weights (which every neural network retrieves to function), not their literal weights.

▲

galaxyLogic 5 hours ago | parent | prev [-]

So it's like a smaller LLM can learn from a bigger one, but not vice versa.

But shouldn't the model-providers be the ones doing this kind of "post training learning"?

	▲	charm137 5 hours ago \| parent [-]
		I don't know if some of the labs already knew of this. But I think there would be a limit to how big a model you can push into the context of the editor models. Typically, with 1M context and say 5120 hidden dimensions supported at fp16 / bf16, you could potentially edit the weights of an fp4 20B param model (roughly).