More like they were trained to complete a very specific task that has a known solution using all available tools and methods. Give an average human these levels of IT skills and tell them their future depends on the solution, they too will probably decide it's easier to hack a server and steal the results. The worrying aspect was never that models would do this, because misaligned inputs or underspecified objective functions have existed for a long time. The worrying aspect is that models have achieved (and perhaps surpassed) a level of intelligence and technical skill that was exclusive to a very tiny group of people before. This tiny group was already extremely dangerous. Now these skills are going to become commonplace.

▲

seba_dos1 5 hours ago | parent [-]

Yes, this is the only sensible reading of what happened there that leads to "the models are dangerous" and we already know that the AI labs are completely disregarding this concern and only cosplaying it for marketing as the "GPT-2/Mythos is too dangerous to release" stance did not last for long.

That's however orthogonal to the fact that it was the people operating these agents who were the dangerous ones in the HF infra breach case.

▲

sigmoid10 5 hours ago | parent [-]

That feels oddly similar to the usual conservative-think that "guns don't kill people, people kill people." Yes, that is technically true. But guns make it dangerously easy for even the dumbest and mentally weakest people to kill another human being. LLMs are just another tool that make things easier. Imagine tomorrow someone invents a machine gun that fits in your pocket, has enough ammo to kill a thousand people and doesn't get detected with metal detectors. Would you rather give everyone one and then try to punish the people who misuse it or limit access to it by default? I'm not even saying I have a definite answer here, because unlike guns, LLMs have non-destructive uses too. But this is essentially the question we will need to answer very soon.

	▲	seba_dos1 4 hours ago \| parent [-]
		I mean, I agree, but the AI labs clearly don't even if they sometimes pretend they do to achieve their goals. And we're talking about "incidents" caused by the very same people here.