Free, and open source models. Now and forever.

jsheard 17 hours ago | parent | next [-]

The problem is that training a free and open source model costs just as much as training a closed one, but has even fewer potential avenues for recouping that investment. The money still has to come from somewhere.

I'm not sure if open weights are immune to being compromised by ads anyway, they can't serve pay-per-impression ads on the output side, but there's nothing stopping the creator from accepting funding in exchange for biasing the training one way or another.

Coming soon: Foobar-600B, a new SOTA open weight model kindly sponsored by Coca Cola, Exxon Mobil and the Heritage Foundation. Please pay no attention to the men behind the curtain.

▲

Adrig 15 hours ago | parent | next [-]

I'm not sure about that. Reports have shown that models from China or Mistral can achieve 80% or more of OpenAI's performance for a fraction of the cost.

If you're tucked in right behind the absolute frontier models, the economics change completely

▲

ACCount37 16 hours ago | parent | prev | next [-]

I would laugh my ass off if Coca Cola Company ends up being the company that solves alignment - so that it can align an "open weight" AI with its corporate interests.

Without that though? Our ability to manipulate LLMs is so shaky I would be really surprised if anyone managed to pull off this kind of model manipulation and have it remain undetected.

	▲	pxoe 11 hours ago \| parent [-]
		I almost believed that they just did, they aren't without their share of quirky and unusual projects and sponsorships.

▲

gldrk 16 hours ago | parent | prev [-]

Just wait until someone leaks an internal SOTA model. Would be deeply ironic given how much AI robber barons ‘respect’ others’ copyright and trade secrets.

▲

justonceokay 17 hours ago | parent | prev | next [-]

What is a free model worth if it’s running on another company’s server farm, trained with data you do not have access to?

▲

Gracana 17 hours ago | parent | next [-]

That is literally the thing the parent poster wants to avoid by running open models.

[edit] I was a little unfair -- lack of access to training data is a bit of an issue (perhaps moreso for analysis than for for actual use, considering what it takes to train these models). I'm thankful that some of them are also distributed as base models, which should be relatively unbiased compared to what happens later during finetuning.

▲

GCUMstlyHarmls 16 hours ago | parent [-]

Run them on what though?

	▲	Gracana 10 hours ago \| parent [-]
		Three power supplies, an old server, a grocery cart and a box fan, and every 3090 you and your friends can get your hands on.

▲

boppo1 16 hours ago | parent | prev [-]

I want models I can run on my machine.

▲

sipjca 17 hours ago | parent | prev | next [-]

I agree, but what about the training data that goes into it (intentional poisoning of the training data, for a variety of reasons, $, power, etc.)

▲

andy99 16 hours ago | parent | prev | next [-]

I’m wondering how long it will be until they are also “sponsored” to have ad content trained in. I personally despise advertising but nobody is building these things out of the goodness of their heart. There needs to be some ongoing incentive to train and release open models.

Similarly, I’m wondering when huggingface is going to need to start showing returns and starts putting ads into transformers etc.

▲

the_real_cher 17 hours ago | parent | prev [-]

To run your own chatgpt level model would require half a million bucks in infrastructure.