| ▲ | hgoel 4 hours ago | |||||||||||||||||||||||||||||||
The HuggingFace incident was the culmination of OAI allowing thousands of agents of various different models - with no clarity on which stages of development they were at (for all we know, some of those models did not have safeguards trained in yet) - to run for at least many weeks without any monitoring in place and with very little thought given to the warning signs (all of the various messageboards) before the incident happened. Theirs was an example of the "reckless waste of resources" I mentioned. We are apparently supposed to believe that OAI takes this incident so seriously as to seek regulation after they have been found to be hiding most of the details of the HuggingFace hack, limiting what their so-called third party investigators can see, and on top of that, had no concerns when they rushed to spin up a 10,000 agent swarm of an internal model, running for several days, to try to get ahead of researchers rumored to have made meaningful progress on a well known mathematics problem. Edit: Actually, we were explicitly told that some of the models used had safeguards relaxed! 'Model-level safeguards were reduced by design. OpenAI said that "deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities"' https://en.wikipedia.org/wiki/2026_OpenAI_agent_cyberattacks... | ||||||||||||||||||||||||||||||||
| ▲ | aswegs8 3 hours ago | parent | next [-] | |||||||||||||||||||||||||||||||
There is nothing that could prevent a bad actor from replicating exactly the same thing with the given goal of e.g. gaining control of critical infrastructure or extorting money. Except for maybe economics. | ||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||
| ▲ | clivefx 2 hours ago | parent | prev [-] | |||||||||||||||||||||||||||||||
What company, product, or period of industrial history do you think met your standard of prudence? | ||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||