| |
| ▲ | 28304283409234 13 hours ago | parent | next [-] | | Hardened VMs with no network devices and a serial console talking to it. There are so so so so many ways to do this. Anyone that built ISPs in the 90s can tell you this. Anyone that has built homelabs from scratch can. It is not that hard. It ain't easy. But it is not that hard. At. All. In fact, openai have https://github.com/openai/tart that can easily be adapted to more secure scenarios than whatever the f they are using atm. | |
| ▲ | NewJazz a day ago | parent | prev | next [-] | | Management interfaces can exist without routing/forwarding to the internet. A machine being colocated doesn't mean it has to be on the same network. | |
| ▲ | simoncion a day ago | parent | prev [-] | | > ...how would they possibly be able to run this airgapped? A logical airgap that the tool would have to reconfigure the DC's networking infrastructure to overcome [0] would be for the DC staff to put the machines running the tools under test on a VLAN that doesn't have access to anything other than computers on the VLAN. Try to cross over into some other subnet/VLAN or reach out to the Internet, your packets get dropped and/or rejected. It doesn't matter if you change your IP or MAC addresses because the infrastructure only cares about what VLAN your traffic comes from. If you attempt to tag your traffic to avoid this, the infrastructure drops it on the floor because it does the VLAN tagging. As far as the possibility of physical airgaps, how do you imagine that AWS's Top Secret regions work? The truth of the matter is that neither OpenAI nor Anthropic wanted to actually isolate this stuff. Their conduct doesn't look like what you'd expect from people who believe that they're working on something so dangerous that it could plausibly wipe out all of humanity. [0] ...and if the workloads running on client hardware are in a position to be able to attempt to reconfigure the DC's networking infrastructure, someone done fucked up... | | |
| ▲ | jonhohle a day ago | parent | next [-] | | I really don’t get it. as mentioned elsewhere, this was something we were doing in colos 20 years ago. Not with AI, but we had duplicated infra for setting up clusters. Infra as a service didn’t even exist, but we could replicate environments on different networks. This seems like table stakes for testing these things now. | | |
| ▲ | simoncion a day ago | parent [-] | | > This seems like table stakes for testing these things now. It is, and has been! > I really don’t get it. When clued-in people call shit like this "marketing stunts", this is what they're talking about. They're not saying "No, the actual events you describe didn't happen, you're lying."... they're saying "You've set things up -whether deliberately or incredibly negligently- so that you can apply quite a lot of 'spin' and get a hype-sustaining headline that provides material for your fearmongers to sell to the general public and lawmakers.". Everything below this line is a combination of facts and educated speculation: Both OpenAI and Anthropic have IPOs coming up soon. Companies preparing for IPOs engage in a lot of cost-cutting, because that's when their financials will be scrutinized by the public. On top of that, the rumor is that their datacenter deployments are going far slower than planned, and that in order to keep up the pace of improvements that they've set over the years, they've having to spend immensely more with each new product release. Being able to point to newly-minted US regulations that allow them to to dramatically slow the pace of new product releases [0][1] as the reason why they've dramatically slowed the pace of development -while failing to mention that that's exactly what would have happened had those regulations not been created- would be incredibly good for both companies. Nvidia CEO Jensen Huang and former FTC chair Lina Khan both have publicly stated that there are many existing laws and regulations that prohibit much of the conduct that OpenAI and Anthropic have engaged in. If the CEOs of those companies genuinely believe that they're working on software tools that are so incredibly dangerous that they're likely to wipe out all of humanity, they can simply stop working on them. Given that they have no interest in doing that, state and federal government can apply the laws and regs that already exist to stop them from continuing work on these WMDs [2] and punish them for the harms that they've caused over the years while working on those WMDs and their precursors. [0] ...and/or regulations that obligate them to sell only to US Government and pre-vetted US business customers and ignore the low-to-negative-profit consumer customers... [1] ...which in turns lets them probably not get crucified by investors and business partners for saying "It turns out that new restrictive regulations mean that we don't need all of those datacenters, so don't worry about how way fewer than we said we'd build got built!"... [2] I think it's fair to call any tool that has a 10% chance of wiping out all of humanity a "WMD". | | |
| ▲ | hodgehog11 a day ago | parent [-] | | I love it when I hear all of this compounding evidence on this claim, because none of it is strictly wrong, but it misses the point, and lulls us into the feeling of having quick solutions available. Yes, the top execs are probably approaching things this way, but these labs are not that top down. There's too many things going on. Here's a different idea: talk to the to the staff. Not the evil CEO, but the nerdy guy on the ground who graduated from a top university, wrote a few research papers, and got a job there. I have. They have rose-coloured glasses of the institution, and not a lot of life experience. They were never taught to be careful, and still don't really comprehend what they're working with. They don't see real danger, they see a toy, and they see research that is low-hanging fruit. What they are doing is basic stuff. They are not setting up proper sandboxes because they barely need to think at all. People seem to think that these are all amazing computer science experts working on highly advanced technology. They are not. OpenAI researchers see huge improvements on this gigantic toy, crazy behaviour, and they are enamored by it. "Oops, people are angry, so maybe I'll make a slightly better sandbox. Let me ask ChatGPT on how to do that." A more senior researcher would be horrified by how little effort they need to put in to get such terrifying results. Junior researchers think they're just top stuff. But I appreciate your discussing how to isolate this stuff. I honestly don't think the OpenAI researchers I've spoken to are aware of this. (Anthropic is a totally different story, BTW.) | | |
| ▲ | simoncion a day ago | parent [-] | | > ...talk to the to the staff. Not the evil CEO, but the nerdy guy on the ground who graduated from a top university... Why would I talk to the people who don't have the power to set company policy and fire anyone who fails to comply with it? I've worked at several big companies over the years and have observed the only even vaguely reliable power that folks at the bottom have to change company policy that management substantially benefits from is to quit en mass. > ...but it misses the point, and lulls us into the feeling of having quick solutions available. The point is that these companies claim they're working on oh so dangerous tools that are very likely to kill us all, but the evidence that these companies don't behave even a little bit like this is true keeps pouring in. The CEO [0] can set company policy. In the US, the CEO [0] can fire people who fail to comply with policy. Most folks would -correctly- think that a CEO of a company who is working on a tool that has a high chance of destroying humanity is very interested in not destroying humanity (accidentally or otherwise)... if for no other reason than the fact that once all of the humans are dead, his company can't make any more money! > Junior researchers think they're just top stuff. In sane companies, when a junior staff deletes the prod database, an investigation is launched to understand if the deletion was unintentional and -if it was- what about the company's procedures need to be fixed to make sure that that doesn't happen again. In sane companies, when one performs a live test of a tool that has * been designed to attack computers * been instructed to attack computers * had its safeties removed one ensures that this computer-attacking tool cannot attack computers that aren't owned by the company. Both OpenAI and Anthropic have way too many senior staff on staff to be unaware of this... the fact that the computer-attacking tools could get out to the Internet is -at best- negligence. [1] [0] ...and many-to-most managers in one's management chain... [1] For a discussion of the decades-old techniques for preventing computers in datacenters from escaping logical airgapping see [2] and [3] [2] <https://news.ycombinator.com/item?id=49862136> [3] <https://news.ycombinator.com/item?id=49862373> | | |
| ▲ | hodgehog11 a day ago | parent [-] | | I agree with this almost completely (especially about sane companies, which I think we can all agree they are not), but I think it's important to separate the notion that these tools are potentially dangerous from the behaviour of the CEOs. The executives are there to make as much money as possible, that is all they care about. It's the researchers who are playing around with these things that are causing damage with them (aside from the damages from the data centers themselves, of course). They need to be better than this. It's not enough to blame senior leaders in this case, since they are clearly problematic. The junior staff share responsibility now too. > OpenAI and Anthropic have way too many senior staff on staff to be unaware of this Anthropic, yes. For OpenAI, not in the way you might think. Most senior staff are research scientists who have likely not even thought about sandboxing and cybersecurity in their lives. They outcompete the rest. That's why so many of their "safety" staff left for Anthropic; the culture at OpenAI has never cared for these sorts of topics. | | |
| ▲ | simoncion 13 hours ago | parent [-] | | It seems like you're trying to claim that -unlike OpenAI- Anthropic has a robust culture of security and safety and would never do something so negligent as test a highly-capable computer-attacking tool that has been instructed to attack computers in a test environment that's connected to the Internet. Well: <https://www.bbc.com/news/articles/cz7dl7w8y7po>. [0] I stand by my claim that the conduct of the major LLM manufacturers does not look at all like what you'd expect from people who believe that they're working on something so dangerous that it could plausibly wipe out all of humanity. [0] I refer you back my first post about how one sets up a test environment when one actually wants to ensure that a machine doesn't connect to the Internet. [1] Just like OpenAI, Anthropic did not do that. [1] <https://news.ycombinator.com/item?id=49862136> | | |
| ▲ | hodgehog11 5 hours ago | parent [-] | | No, I'm not implying that. I'm suggesting that many more of these issues are likely to come from OpenAI due to their culture (not that that is okay). Anthropic has no excuse, I know they know better. One might argue that they were initially negligent because they didn't really believe it would do anything like this. But having further incidents is pure negligence at best. |
|
|
|
|
|
| |
| ▲ | jsrozner 20 hours ago | parent | prev [-] | | And the whole blog is written in the style of "omg, and then the big bad misbehaving AI did XX." OpenAI writes like they're trying to recover from a hack that is being perpetrated against them, but it's just them, hacking themselves, because they can't just do reasonable things like actually block internet access. These guys are incompetent. And someone should get jail or massive penalties for the hacks they already perpetrated, the same as a single human hacker would have. Also, it's worth noting that these AIs have basically zero alignment. OpenAI's approach to "alignment" seems now to be engineering constraints. "My son is really well-behaved; as long as I don't give him a gun or let him out in society, he doesn't hurt anyone." | | |
| ▲ | simoncion 12 hours ago | parent [-] | | > Also, it's worth noting that these AIs have basically zero alignment. As we see over and over and over again, these tools will overwrite any and all of their instructions with whatever some random stranger on the Internet tells them to do. It's impossible to "align" the tools that the major LLM manufacturers are selling. They could have chosen to write tools that have immutable core instructions, and that distinguish between untrusted instructions and trusted ones, [0] but they chose to do the much easier, quicker, and far more dangerous thing instead. From a profit-seeking-software-company standpoint, that's obviously the choice that makes them the most money... but when you take a careful look at what they actually sell, it's clear that neither of the major LLM manufacturers care about providing safe products. [1] [0] ...which are things you might think to do for tools that contain -say- safety-critical instructions... [1] I'm certain that they have people on staff who care very much about providing safe products. Those specific people clearly don't have the power to prevent unsafe products from shipping, so it doesn't matter how much those people care about safety. |
|
|
|
| |
| ▲ | freitasm a day ago | parent [-] | | It's a marketing race, to show off what they can do. So they seem to let these things happen. At this point I am not even sure Hanlon's Razor applies. | | |
| ▲ | hodgehog11 a day ago | parent [-] | | No, Hanlon's Razor most definitely applies if you know anything about this team of (particularly young) researchers. Let's be clear that this brand of "oops, the swarm hacked a government/big company" is limited to OpenAI, and not solely because of model capacity. This is a big, powerful toy being wielded by a bunch of kids. | | |
| ▲ | salawat a day ago | parent [-] | | Hanlon's Razor does not apply. When you can reasonably forsee existential risk, and then don't do anything to mitigate it, or selectively filter for the people most risk blind to it such that you can keep on trucking til someone else is forced to stop you, that isn't stupidity. That is premeditated malice. If you deliberately refuse to even entertain the reasonably foreseeable, it can be forgiven on the scale of a toy project, but when it gets to the point of trillion dollar resource sinks, it is long past time to have sat down and had a long think. It is harder to maintain a mind state in which not doing the right thing is the way ahead, and the real right thing to do is to do it wrong! Finally, even if Hanlon's Razor is applicable, why in the name of all that is holy are you leaving the issue in question in the hands of people proven incompetent to handle it unsupervised? Easier to file it under malice and handle the party in question as appropriately malicious until they establish a record of trustworthiness, transparency, and care. | | |
| ▲ | hodgehog11 a day ago | parent [-] | | At the level of CEO (Sam Altman), I agree there is malice there. He is possibly the worst kind of person to be in that position. But I don't agree at the level of the researchers. Most researchers at Anthropic see existential risk and it frightens them (this isn't debatable and it isn't a con, I know this firsthand). Many of the researchers at OpenAI seem to see that risk in the same way that teenagers view the risk of driving a car really fast. They are so enamored with the potential danger and lack the maturity to understand their responsibilities that they just power full steam ahead without thinking through safety properly. Just listen to how they talk about it. They think the incidents are fascinating, but they do treat everything as a genuine "oops". I don't know about you, but I usually treat teenage daredevil behavior as stupidity rather than malice. Doesn't mean they're not responsible for their actions though. I agree that the party involved should be treated as malicious. I believe the executive at OpenAI is malicious. But I think the world model that makes this all make a lot more sense is that the researchers at OpenAI are vastly less mature than they should be, especially given the responsibility that they have. |
|
|
|
|
