Remix.run Logo
▲ Toutouxc 6 hours ago

I’m hoping someone can explain this to me. They say “no training”:

> for the first time, a language model rewrites its own weights by its own reasoning and becomes better, with no gradients and no training

> it tests its edits on practice problems and submits only what works, each edit improving on the last, and over eight consecutive sessions its accuracy only rises

But this sounds a lot like training, and “submitting only what works”, while probably not gradient descent by definition, also seems kind of obvious when you’re optimizing something.

▲ozgung an hour ago | parent | next [-]

I have been working on weight-space-programming as a side project since the blog post. Here is what I understand:

- We normally train deep neural networks with back-propagation. We compute gradients to find out how to tweak the weights toward the desired output.

- That's a fixed recipe. And it's not the only way to tweak the weights. It's just the best one that works in scale. The brain, for instance, does not do back-propagation, as far as we know.

- There are other known mathematical methods (mostly from Linear Algebra) to analyze the activations and compute what weights to change and how, to push the output to the desired way.

- A sufficiently advanced LLM knows about these methods and can analyze data like activation maps. With a specific target in hand, it can combine them into new constructions. (see [2] for some of the methods they used and I collected in a repo)

- So it's not a fixed training for-loop anymore. It's a dynamic "training" iteration. The LLM looks at the model's outputs, reads some of its layers on chosen inputs, computes an edit, applies it and tests it.

- It's dynamic because LLM chooses that to do and how to do it at each iteration by reasoning. I think that's the most interesting part and could be a new paradigm.

- I also compared it with gradient training. On my target, training was better (details in [1]), but WSP can find its niche. Its biggest advantage is that it needs no backward pass: less memory, and the model only has to run inference. It needs no differentiable loss, it can enforce constraints directly instead of through a penalty term, and the edit is easier to interpret.

[1] https://medium.com/@ozgungenc/weight-space-programming-on-a-... [2] https://github.com/ozgung/weightprog/blob/main/docs/methods....

▲fl1pper 5 hours ago | parent | prev | next [-]

The paper states that this approach doesn't use backpropagation, an optimizer, or a parameter-fitting loop.

What makes it work is that it reads its own hidden states and weight matrices -> then writes a few-line Python script to fix the weights directly (within a 20-forward-pass budget, so it can't just brute-force guesses).

After skimming through the paper and comparing it with how SEAL works (where the model still relies on RL and gradient descent to apply its self-edits), I understood it something like that :)

But I think there are comments in the thread that explain the main concept far better than I do.

▲ex-aws-dude 2 hours ago | parent [-]

Wouldn't you get crazy overfitting?

▲Tuna-Fish 6 hours ago | parent | prev | next [-]

What it means is that they are not doing backprop.

▲joe_the_user 5 hours ago | parent [-]

The researchers aren't doing backpropagation and instead calling the model and asking it to adjust its weights to improve itself and the model isn't using external tools. Of course, it has been shown that models learn to do backpropagation internally as part of their training. Some it may well that they are indirectly doing backprogation - or they could do one of the other algorithms that LLMs are shown to learn as part of their training.

I think the real question is going to be whether this is more efficient than just conventionally training the model.

▲William_BB 5 hours ago | parent | prev | next [-]

Is it actually converging to something meaningful? You could write the same procedure in a simple deterministic python script, that doesn't mean it's not learning garbage in the long run.

▲anon291 5 hours ago | parent | prev | next [-]

Training means we take the mathematical expression corresponding to the entire model, use differentiation rules to generate partial derivatives according to a user-specified loss function (not part of inference), and then apply the gradients from the calculated differentials using a learning rate.

In this case, they just ask the model what the weights ought to be.

▲jiajunf3 3 hours ago | parent | prev [-]

[flagged]