What anyone paying attention can see is that scaling is obviously hitting diminishing returns.

> The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks.

This sentence is entirely based on unverified accounts from OAI. They haven't released logs or let anyone outside the company (who doesn't have life changing options in OAI) verify anything. Huggingface can only verify that the hack happened and that it had the hallmarks of an AI agent. Was the agent assisted and directed by humans within OAI that really wanted to put the competition into stasis? Did the agent really escape or did someone at OAI leave the prison door open?

OAI has watched all the same movies you have an they are relying on those movies causing us to blindly regulate before actually asking basic facts about what actually happened.

Do you know the story of the boy who cried wolf?

There may very well be a wolf lurking [0] but OpenAI/Anthropic have both cried wolf so many times, incorrectly, that it’s incredibly hard to believe “this time there IS a wolf!”. Remember “GPT-2 is too dangerous to release”?

I had a conversation at work just yesterday about how we need to start hardening things we’ve let languish because of the coming LLM-backed attacks we are sure to face, even if just from a script kiddy. I do think we are headed in that direction, however it’s Sam/Dario’s own fault that people aren’t going to take them seriously.

Lastly, as other have pointed out, this seems more financially motivated than our of any real desire for “safety”. We’ve all seen how both labs approach “safety” so it’s quite rich for them to now hide behind that after not giving a shit before.

[0] I don’t take anything Sam or Dario say at face value. The whole hacking thing could also be a case of them letting a model loose on purpose for the publicity, not an “escape” during a training run (or whatever they said). And when both, especially Sam, have lied so much and breathlessly warned about the dangers of AI (when it helped their bottom line and/or helped pull up the ladder behind them), it makes it hard to believe them.

You see it as good news, I see it as writing on the wall that they are losing control. These actions won't scale for more powerful models. We knew the frontier was going to be dangerous, it is, it only gets more dangerous from here, and no one cares until it's too late.

What about Mythos 3. Are you willing to make that bet?

You can’t say something is world ending without it actually ending the world otherwise you’re a liar - a bit of a catch 22 there.

By that logic the model that ends the world won’t be called world ending at first. Is that a game you want to play?

I don't disagree about the danger. But if it is dangerous, why isn't OpenAI opening dialogues with all the labs and politicians across borders to basically say "we need to stop now"? Cyber models are constrained by the total compute and electrical capacity of the globe. We are still at a point where it is impossible to build an agent with offensive capabilities in the basement. We can effectively track and trace capabilities if we had the political will to and could for some time while we build more effective processes to prevent a malicious actor from doing so. These things have tremendous compute and energy requirements and don't scale like old school software does. We could absolutely do it.

But no, that's not what OpenAI is saying. They haven't put up the actions that would earn them that trust. Indeed they've driven the world and whatever capital they can get their hands on straight to this precarious cliff.

So you're right, the danger is real. But the solution starts with removing the men who had their hands on the steering wheel to get us this far. Any other action is disingenuous unless they pull a miraculous 180 in their ethics.

In other words, when the bully plays "why are you hitting yourself?" you don't listen to the bully's solutions, you restrain the bully.

Your argument is essentially: "I made a claim and presented extremely weak evidence (sci movie plots and unverified claims from ultra conflicted sources). You rejected this evidence as insufficient. Therefore no evidence will ever satisfy you. Therefore I don't need to produce any evidence. Therefore my claim is true."

What would the logs show? They would show what actually happened.

What would a public demonstration that experts without billions in options could evaluate show? It would show actual danger.

What would publicly having your compete in controlled and legal hacking competitions show? Actual danger.

This is not a high bar of evidence.

Do you actually think a sci fi plot and OAI press releases are all the evidence you need? Because if that's true then I hope you haven't watched Independence Day or 28 days later.