training is modifying the weights. How you modify them is not the object of a license, never was.

▲ v9v an hour ago | parent | next [-]

Would you accept the argument that compiling is modifying the bytes in the memory space reserved for an executable?

I can edit the executable at the byte level if I so desire, and this is also what compilers do, but the developer would instead be modifying the source code to make changes to the program and then feed that through a compiler.

Similarly, I can edit the weights of a neural network myself (using any tool I want) but the developers of the network would be altering the training dataset and the training code to make changes instead.

	▲	NitpickLawyer 24 minutes ago \| parent [-]
		I think the confusion for a lot of people comes from what they imagine compilation to be. In LLMs, the process is this (simplified): define_architecture (what the operations are, and the order in which they're performed) initialise_model(defined_arch) -> weights. Weights are "just" hardcoded values. Nothing more, nothing less. The weights are the result of the arch, at "compile" time. optimise_weights(weights, data) -> better_weights. ---- You can, should you wish, totally release a model after iitialisation. It would be a useless model, but, again, the license does not deal with that. You would have the rights to run, modify and release the model, even if it were a random model. tl;dr; Licenses deal with what you can do with a model. You can run it, modify it, redistribute it. They do not deal with how you modify them (i.e. what data you use to arrive at the "optimal" hardcoded values). See also my other reply with a simplified code example.

▲ noodletheworld an hour ago | parent | prev [-]

> And most importantly, you modify the work just like the creators modify the work

Emphasis mine.

Weights are not open source.

You can define terms to mean whatever you want, but fundametally if you cannot modify the “output” the way the original creators could, its not in the spirit of open source.

Isnt that literally what you said?

How can you possibly claim both that a) you can modify it the creators did, b) thats all you need to be open source, but…

Also c) the categorically incorrect assertion that the weights allow you to do this?

Whatever, I guess, but your argument is logically wrong, and philosophically flawed.

▲ NitpickLawyer 42 minutes ago | parent [-]

> Weights are not open source.

If they are released under an open source license, they are.

I think you are confusing two concepts. One is the technical ability to modify weights. And that's what the license grants you. The right to modify. The second is the "know-how" on how to modify the weights. That is not something that a license has ever granted you.

Let me put it this way:

```python

THRESHOLD = 0.73214

if input() < THRESHOLD:

  print ("low")

else:

  print ("high")

```

If I release that piece of code under Apache 2.0, you have the right to study it, modify it and release it as you see fit. But you can not have the right (at least the license doesn't deal with that) to know how I reached that threshold value. And me not telling you does not in any way invalidate the license being Apache 2.0. That's simply not something that licenses do.

In LLMs the source is a collection of architecture (when and how to apply the "ifs"), inference code (how to optimise the computation of the "ifs") and hardcoded values (weights). You are being granted a license to run, study, modify and release those hardcoded values. You do not, never had, never will in the scope of a license, get the right to know how those hardcoded values were reached. The process by which those values were found can be anything from "dreamt up" to "found via ML". The fact that you don't know how those values were derived does not in any way preclude you from exercising the rights under the license.

	▲	roblabla 17 minutes ago \| parent [-]
		You are fundamentally conflating releasing a binary under an open source license with the software being open source. Nobody is saying that they're violating the license of Apache2 by not releasing the training data. What people are objecting to is that calling this release "open source", when the only thing covered by the open source license is the weights, to be an abuse of the meaning of "Open Source". To give you an example: I can release a binary (without sources) under the MIT - an open source license. That will give you the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of said binary. In doing so, I would have released the binary under an open source license. However, most people would agree that the software would not be open source under the conventional definition, as the sources would not be published. While people could modify it by disassembling it and modifying it, there is a general understanding that Open Source requires distributing the _sources_. This is very similar to what is being done here. They're releasing the weights under an open source license - but the overall software is not open source.