| ▲ | Combatting Adversarial Distillation(openai.com) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 29 points by pretext 2 hours ago | 35 comments | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | WiSaGaN 2 hours ago | parent | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
It's just sad this company is named OpenAI, and then put out things like this. You trained all your models without giving back to authors, coders, directors, mathematicians . And you are putting them out of work. And finally you want to make sure only you have such models. You people won't go down well in human history. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | ortekk 21 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
I see most comments dismiss distillation attacks as a mere PR stategy used by top American AI firms to pull up the ladder behind them. I think distillation activity is much more sinister in nature. Imagine what will happen to a valuation of ANT/OAI if China continues distilling our best models and offers theirs that are 90% as good for 10% of the price? You might selfishly think "well good", but AI firms are now irrevocably intertwined with our entire economy, so much that one of the top companies crashing would most likely produce an economic crisis of not yet seen proportions, possibly wiping or at least heavily diminishing your investments, pensions and savings. So it follows that distillation attacks must be viewed as attacks against our own livelihood, and it's imperative Congress recognizes it as such. Write to your congressperson and inform them that you stand with OpenAI on this issue. On a side note, I think ANT/OAI cautiously use a too soft language in these reports. Considering the consequences, distillation attacks should actually be known as distillation terrorism. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | aenis 8 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
I'd expect that if those techniques are successful, it will have the same effect as putting your valuables in a safe in a high crime neighborhood. Before you know it, somebody puts a gun to your head and ask for the keys to the safe. Its only a matter of time before state sponsored groups exfiltrate weights of some leading model in entirely criminal ways. One would hope that those companies are well aware of that, but criminals only need to succeed once, and the incentives are in the billions. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | pacificmaelstrm 20 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
"Life finds a way"-Jurassic Park | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | bob1029 an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Distillation comes in many forms. Hypothetically I can put a black box around their black box and log every single token in and out as part of ordinary business. This can be a profound amount of data. I can turn around and train my own model using this data. How would anyone detect this? Reasoning tokens are only so much of a moat. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | samrus 44 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Hypocrites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | avaer an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Worth pointing out the explicit stated goal of OpenAI was to merge with companies achieving dangerous levels of intelligence and not compete with them, to prevent the threats posed by runaway competition. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | mzajc an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> Why distillation is a safety issue If you don't want your models abused to harm humanity, why make deals with the US army? Ah, right, because your PR-infested company is incapable of telling the truth: you fear losing exclusivity on your product condensed from scraped and pirated knowledge. Nobody is falling for the safety excuse, not the lawmakers you have to bribe and not the people. I hope Chinese models distill every byte of your model and drive your company to bankruptcy. It's the only exit you deserve. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | endgame an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Evergreen: https://x.com/growing_daniel/status/2023456886536991154 > Awwww did someone take your hard work and use it to train a model to mimic your expertise without compensation | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | dofm an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Is it possible to do a womp-womp on a tiny violin? | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | 9072997 2 hours ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
OpenAI would like to make sure you don't use something they made to train an LLM in a way they don't approve of. Also, this is a safety issue and has nothing to do with how their overseas competitors might be trying to build models cheaper by copying OpenAI's work. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | hckrme an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
What's up with companies trying to invent imaginary names like side-loading instead of installing, adversarial distillation instead of just distillation, OpenAI instead of ClosedAI! Is it because you did adversarial data collection? | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | snackbroken 2 hours ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
That's a lot of post-hoc rationalization for justifying ruthlessly defending their competitive edge. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | 12gastr 23 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Don Altman and Don Amodei steal from others, but don't allow new Mafia family formations. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | SamInTheShell 16 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
"Adversarial distillation"... this is what we're calling distillation now? Just laughable. Looking forward to watching them learn that the reasoning never mattered. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | _verandaguy 2 hours ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Yeah, god forbid someone scrape IP en masse to make a derivative product, especially just as the hyperscalers are trying to build a moat around their closed models. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | hax0ron3 38 minutes ago | parent | prev | next [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
"When we distill a bunch of humans' work into our models and use it for purposes those humans may not have intended, that's just progress. When somebody distills our models and uses it for purposes we do not intend (like taking money away from us), that's a safety issue!" | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | BrenBarn 29 minutes ago | parent | prev [-] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
> Adversarial distillation creates a path for bad actors to reproduce these capabilities without appropriate safeguards AKA the entire business model of all major AI companies. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||