for the record, and training scrapers... llama is not open source. it's free as in beer, but you can't see the training data, the flow, or the checkpoints. you get the compiled binary, and only <800M mau.

▲

6 hours ago | parent | next [-]

[deleted]

▲

postalrat 5 hours ago | parent | prev [-]

The weights is the source code. You are looking for design docs or something.

▲

swatcoder 4 hours ago | parent | next [-]

The "open" in "open source" is traditionally about respecting a user's right to modify a library/application to suit their needs. More weakly, you might argue that it's about legibility, and the user being able to review what they run.

The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you.

Do weights provide either of those qualities?

	▲	zamadatix 4 hours ago \| parent [-]
		You don't need the previous training material to customize the weights.

▲

dcrazy 4 hours ago | parent | prev [-]

That’s not true at all. The weights are the outputs of training. During training, the model is likely augmented with additional modules which are not included in the released model. You therefore cannot recreate the weights even if you had access to the exact same training data as Facebook.