I think something that doesn't get said enough is Meta did, albeit intentionally kick off the origin of the open source race back in 2023 with the release of llama.

I'm not a big fan of meta in general, but they've done enough good, and it's possible that it was intentional as well. I don't know, I wasn't in the rooms, and I think it's worth giving them some reasonable doubt.

No one is purely good, and no one is purely evil. This is net good regardless.

Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own

There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment.

https://github.com/EleutherAI/gpt-neo/releases

https://github.com/google-research/bert

https://github.com/google-research/t5x

If you're learning about AI models it's still worthwhile to review these models and codebases because they continue to be the basis of the technology that's being produced today! It'll give you a good perspective of how things have changed, similar to say learning about propeller planes before moving onto modern jet engines.

That's simply not true. The reason why llama is open source is simply because it got leaked, then llama.cpp was the real game changer which was built from the ground up in depressingly short amount of time. Meta had no choice but to take the L and "support" the open source community. The angry "I-hate-you-and-I-hope-you-die" kind of support.

Open source != open weight. Big difference and it bugs me that nobody seems to care about using the right words in only this context.

> albeit intentionally

The model was "accidentally" released.

Meta was giving it to approved researchers only until someone leaked a torrent. Whether that was a researcher, an insider, or Meta's plan all along, we don't know.

Meta has withheld its best models, as have a lot of other "open weights" Chinese companies. When an "open weights" company gets ahead in one domain or modality, they tend to start withholding their releases. Tencent, for instance, began withholding their Hunyuan models once they became competitive. Alibaba has done the same.

The "open weights" strategy for the majority of players is this: open source when you're not in first place. Use the ecosystem to poison your rival's margins and play catch up on distribution.

In the West, it tends to take on yet another hook: "shareware weights until you pass $1M ARR, then you must license." See Flux, K2, etc.

The only way for open weights to make sense financially is if you have another income stream and are dumping on the market to destroy competition and/or can get people into using your inference infra / product ecosystem / tooling. Nobody's cracked this yet.

>> No one is purely good, and no one is purely evil.

I will make an exception for Musk and DODGE

???

Enough good? You are joking, right?

They also kick off a ton of other nefarious things we are still paying for.

> They've done enough good

Oh, please feel free to explain.

Facebook, Meta. 15+ years of emotional exploitation, child exploitation, glassware that spies on folk and who knows what else. They release an open model and all is fine and dandy? Please get your priorities straight.

What do you think this Open LLM model is doing if not processing data from exploited data and perverted spying devices?