| ▲ | wcoenen 5 hours ago |
| This post does not mention safety at all. What's to stop bad actors from fine tuning open weights to run fully automated genius-level scams personally targeting basically everybody? |
|
| ▲ | bigbadfeline 5 hours ago | parent | next [-] |
| > What's to stop bad actors from fine tuning open weights to run fully automated genius-level scams personally targeting basically everybody? You mean like ChatGPT hacking Hugging Face? Obviously nothing can stop the closed weights providers from doing "genius-level scams" and in addition you won't know how they did it and what models were used. In short, only a good guy with open weights can stop the bad guys with closed weights, be them fine-tuned or pre-trained. |
| |
| ▲ | wcoenen 3 hours ago | parent [-] | | The OpenAI model that broke out of its sandbox and hacked HuggingFace was running without guardrails. To quote the OpenAI post[1]:
These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities. At least OpenAI will make an attempt to fix their sandboxes, and will not give the public access to models without guardrails. Open weight models on the other hand will run without guardrails almost by definition. I don't think it's wise to provide those capabilities to scam call centers. [1] https://openai.com/index/hugging-face-model-evaluation-secur... |
|
|
| ▲ | Larrikin 5 hours ago | parent | prev | next [-] |
| What's stopping someone from doing that right now without the LLM, just slightly slower? |
| |
| ▲ | gogopromptless 5 hours ago | parent [-] | | It's perfectly natural for a prey population to experience a crash when a new predator enters the ecosystem. What's quite odd is a prey species is manufacturing predators in some sort of reverse evolution, where we started out as symbiotic and are industriously pushing towards full parasites/predators, but I guess life finds a way. |
|
|
| ▲ | kingstnap 5 hours ago | parent | prev | next [-] |
| Whats to stop me from going down to the local gas station, filling up a few thousand liters of gas in a rented moving truck, and driving into my nearest hospital? Most people aren't terrorists. You can't just argue stuff is dangerous because it can be one piece of a sophisticated plot. |
| |
| ▲ | throw1234567891 4 hours ago | parent | next [-] | | Nobody will sell you a few thousand litres of gasoline at a gas station. | | |
| ▲ | kingstnap 4 hours ago | parent [-] | | You don't have to buy it all at once all in one place. A sophisticated actor can accomplish a lot. You can't bubble wrap the whole world. The real thing to care about are the incentives and having strong morals and norms. Something that makes me deeply concerned about the US going full mask off recently. | | |
| ▲ | SCHiM 4 hours ago | parent [-] | | I think the attempt of people to "bubble wrap" the whole world actually makes things worse. Treat people as untrustworthy criminals by default, and guess what you get by default? |
|
| |
| ▲ | chriswunan 3 hours ago | parent | prev [-] | | In China, you can't get that much fuel in local gas station to fill up a truck. That is just one of many things you can't do in China. In the name of safety. Keep asking that question will probally end up with fewer and fewer things we can do. |
|
|
| ▲ | _aavaa_ 5 hours ago | parent | prev | next [-] |
| Amazing how we quickly the crypto wars have been forgotten and the lessons not learned. |
| |
| ▲ | marknutter 3 hours ago | parent | next [-] | | Begun, the ai wars have. | |
| ▲ | abernard1 4 hours ago | parent | prev [-] | | "But what about criminals?! What about terrorists?! Won't someone please think of the children?!!!" Every argument against open source AI is moot. Outside of totalitarianism, you cannot control people building algorithms with math and software. And even that won't work in the long run. People are not going to stop their AI printing presses because the Church of Venture Capital needs regulatory scarcity to command valuations. | | |
| ▲ | iamnothere 4 hours ago | parent [-] | | > Outside of totalitarianism The flaw in your argument is assuming that decision makers don’t want this | | |
| ▲ | abernard1 4 hours ago | parent [-] | | Ah. But which decisionmakers? Would it be the ones that tried to backdoor SSL and encryption in the 90s? Would it be the ones who argued for censoring speech on social platforms? Do those individuals in both cases share the same political ideology as the Bay Area? I'll provide a counter thought experiment. What if, a certain politician whose campaign was derailed by people who censored him and kicked him off every tech platform simultaneously... thought it would be very funny to "commoditize his complement"? |
|
|
|
|
| ▲ | chriswunan 3 hours ago | parent | prev | next [-] |
| It's already an established industry with human resources. If you read news about how these scam factories operate in Southeast Asia, their cost to use real humans is even cheaper than running a 2.8T model locally for scams. You don't need genius-level intelligence to run scams. Maybe we tend to overestimate how smart people are. |
|
| ▲ | SimianSci 5 hours ago | parent | prev | next [-] |
| How can we close pandora's box, now that its open? Safety is not restricting access to only people favored by the government. The greed of AI executives opened pandora's box. Now they are desperately trying to find ways to reap the benefits with none of the consequences. |
|
| ▲ | BraveOPotato 5 hours ago | parent | prev | next [-] |
| I don't believe trusting big brother and big tech is the solution either. Besides, even if it's open weight, it still runs on someone else's hardware. Unless you have the money to run them locally, and if you do, you could do a lot worse than scams. Ask any lobbyist. |
|
| ▲ | trollbridge 5 hours ago | parent | prev | next [-] |
| Nothing, which is already happening with weaker models which are already released. |
|
| ▲ | Barrin92 2 hours ago | parent | prev [-] |
| their reputation, future and business and the fact that people can go somewhere else, just like anywhere else in the open source world. What stops RedHat from putting genius level scams into their operating systems? The fact they're done after they'd pull of their first scam. Obviously if you're using something anonymously published to the internet you can get scammed the same way you can get scammed on Craigslist, so just don't do that |
