Why should we believe that a scaled out version of something that happened a few months ago "simply cannot happen"? How many dollars of compute do you believe were available to the swarm(s) behind the OAI-HF, German wiki, and Rubygems incidents?

▲

anon84873628 10 hours ago | parent | next [-]

Well a big reason that we criticize OpenAI for that is because they were the ones giving it access to the massive compute necessary for the LLMs to think. If they had been responsible about their experiments or what types of workloads they allow their LLMs to operate, it wouldn't have happened. Very few companies could enable those workloads.

	▲	muvlon 6 hours ago \| parent [-]
		Well sure, but access to that compute is gated by simple credentials (like API tokens). Those can be hacked. Imagine for example if a model hacks into ~every Linux computer on the internet using an 0day and steals their OpenAI, anthropic and openrouter credentials. It now has access to billions of compute and the only way to fully stop that is for multiple major providers to shut down services entirely. That's already well into "billions of dollars of damage" territory.

▲

pr337h4m 10 hours ago | parent | prev [-]

Do you realize how big "the entire internet" is?

> How many dollars of compute do you believe were available to the swarm(s)

At least two OOMs more than the dollar value of the damage they'd caused. (Also, as an aside, IIRC, the wiki servers weren't breached; it was just a lot of spam.)

	▲	kalkin 9 hours ago \| parent [-]
		Sure. And there's an OOM more compute coming online in the next year or two, while models at a given capability are getting cheaper. "Two OOMs" of scale relative to the HF swarm seems like a bit of a red herring to me, but also within the realm of possibility. The Internet is big, but one can do quite a lot of damage with ordinary bots and worms that exploit individual widespread vulnerabilities, which LLMs are perfectly capable of writing. Most of the damage also doesn't rely on hitting every long-tail website. I'm honestly not that concerned about cyber impacts of LLMs relative to other impacts. I just don't like to see the whole concept of being worried dismissed as obviously baseless on the basis of one pretty shaky scale argument.