I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model.

They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?

▲ WithinReason 3 hours ago | parent | next [-]

If they open sourced the training data and code but you had to train the network yourself, would that be open source?

▲

croemer 2 hours ago | parent [-]

Yes, obviously.

▲

swiftcoder 2 hours ago | parent | next [-]

Even though none of us could actually afford to train it?

▲

croemer 2 hours ago | parent | next [-]

Some of us might be rich, or get funding, or in the future it could be useful when training is cheaper.

Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it.

▲

gpugreg 2 hours ago | parent [-]

> Some of us might be rich

I sure wish I had a few 100M of disposable income to train a frontier model.

> or in the future it could be useful when training is cheaper.

I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

▲

Muromec 2 hours ago | parent [-]

If you would tell people at the start of 20th century how much energy we consume, they may not believe you or think it is wasteful.

	▲	gpugreg an hour ago \| parent [-]
		Looks like global energy consumption has risen by an order of magnitude from 1900 to 2000: https://www.encyclopedie-energie.org/en/world-energy-consump... Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.

▲

eptcyka 2 hours ago | parent | prev [-]

Maybe not now, but what about 10 years down the line?

▲

WithinReason an hour ago | parent | prev [-]

As an example, Grok 4 took $388M to train

▲

HPsquared 24 minutes ago | parent [-]

Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights.

The weights are literally a binary blob.

	▲	WithinReason 6 minutes ago \| parent [-]
		The weights are the modifiable representation. You modify them with gradient updates.

▲ mlazos an hour ago | parent | prev | next [-]

The models are open source, it’s never been a requirement for someone to document the entire process of creating something to be “open source”. Just sharing the source, in this case, the weights, meets the criteria. Software engineering’s obsession with precise terms is actually one of the things I’m glad is dying with ai automation.

	▲	andy99 an hour ago \| parent \| next [-]
		> obsession with precise terms They’re not really even being precise. The relevant software freedom, from the FSF is [] `The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.` Ported to the model world, this is fulfilled by sharing the weights and implementation. There’s almost nothing that having the training data gets you (other than actually training it). The weights plus a reference implementation let you see all the states to study the behavior, and let you fine tune it to do your bidding (the abliteration etc). The freedom is satisfied. Some might argue that without the training data you couldn’t do some classes of experiments to see how it works, say leave-one-out retraining. I’d argue things like that are not really about the model but about ML research or the class of models, which while interesting is not a free software pre-requisite. [] https://www.gnu.org/philosophy/free-sw.html#four-freedoms
	▲	HPsquared 22 minutes ago \| parent \| prev \| next [-]
		Model weights are literally a compressed blob of binary data. The end result of a compilation process.
	▲	nextaccountic 36 minutes ago \| parent \| prev [-]
		The source is actually all training data, plus the software used for training, including some scripts or instructions to run the entire thing end to end on your own computer. That's what open source has always been about. The weights are the output of a program, it's a binary. It's not source.

▲ stavros 3 hours ago | parent | prev | next [-]

Agreed, it's "open weights". Open source would be to publish the entire process so you could tweak it if you wanted.

▲ charcircuit 4 hours ago | parent | prev [-]

The weights are the preferred form for modifying or integrating with other models. There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.

>a term that it seems nearly everyone has agreed is appropriate

Models being considered open source even if the original training code / data is not released also is something almost everyone has agreed to be appropriate.

▲

dietr1ch 3 hours ago | parent | next [-]

> The weights are the preferred form for modifying or integrating with other models

It's the 2nd time I hear this argument and I'm already fed up with it

Is it the preferred way only because training is expensive? It's like saying binaries are the preferred way of modifying program because you can't afford to have a fast enough machine to compile it yourself.

Most people don't have resources to compile their own browser, but what makes some browsers open is access to the source.

Maybe it's the preferred way because the people sharing models are themselves working with weights and no data?

	▲	NitpickLawyer 3 hours ago \| parent [-]
		> Is it the preferred way only because training is expensive? No, it's the preferred way because that's literally how you train it. Contrary to popular misconceptions, you don't "compile" data into weights. You initialize a model (based on architecture, config, etc) and then you modify it via training. But crucially they (i.e. model creators) modify it the same way (technically speaking) as you would. That's what the license grants you. Nothing less, nothing more. The "how" as in knowhow has never been something covered by a license. Open-weight is something dreamt up by people misunderstanding the basics of model creation and training, and having ideological biases against AI and/or LLMs. It is what it is, but you should know that you are technically wrong. A model released under an open source license (Apache, MIT, etc) is an open source model. Because the weights are the source of the models. Training is not "compilation". Training is the "how" as in knowhow to modify the model. Training deals with values. Source deals with everything, including values. In the past, if someone would have released a piece of software (say a PID controller algo) w/ hardcoded values, no-one would bat an eye. LLMs are just that, with billions of hardcoded values. Nothing less nothing more.

▲

teruakohatu 4 hours ago | parent | prev [-]

> There is no obligation in open source to transitively open source all of the documentation / tools used to create the open source project.

Open source means open source code. Open weight means a binary file dump, not unlike an exe file. There is nothing open source about it.

Its like having a closed source text editor that censors certain words, and an open source text editor that censors certain words.

The latter can easily be recompiled, the former requires reverse engineering. Both may give you a license to use them freely.

▲

Alpha3031 3 hours ago | parent | next [-]

Per the OSD definition of source code, "the source code must be the preferred form in which a programmer would modify the program." which means an argument could be made (as charcircuit is making) that the weights, being the preferred form to modify, are the source.

I do prefer open weights as being more precise (like, is it even really software that has source code in the first place?) but I feel like at this point the ship has sailed somewhat (though if this is something you're willing to spend your time arguing then... moral support I guess?)

▲

ciupicri 2 hours ago | parent [-]

You can also edit/patch binary executables, but no one says about that software that is open source.

	▲	Muromec 2 hours ago \| parent [-]
		You can, but its not the first preference

▲

charcircuit 2 hours ago | parent | prev [-]

I think you are failing to see how the weights are the preferred form of working with a model. It's like if I shared a png with others to use as a meme template. Even if I don't share the source code to photoshop other people can use that "binary file dump" to make derivative images of their own with it.