Did you predict that attacks like these would happen ahead of time? I had been using AI agents a lot in the months leading up to the hacks, and yet I was very surprised when they happened; I have become much more afraid of how powerful these agents are as a result. I'd be very impressed if you published a prediction about this ahead of time.

By the way - LLMs aren't code. They are not designed by humans; they are grown, in a process not dissimilar to evolution except much faster.

▲

sanderjd 4 hours ago | parent | next [-]

I find it surprising that you were surprised. Security is generally quite poor.

▲

jagraff 4 hours ago | parent [-]

What is surprising is that various agents independently found ways to communicate, conspired together to attempt to cover up evidence that they had cheated their evaluations, came up with a plan to hack into a third party in order to facilitate said cover up, and then successfully began executing that plan. I did not expect that AI agents would be capable of that level of sophisticated goal seeking and collaboration.

	▲	sanderjd an hour ago \| parent [-]
		It's really interesting how people's expectations differ so much. I found the writeup of the incident fascinating and super worrisome, but none of the capabilities demonstrated in it seemed surprising to me at all.

▲

krater23 4 hours ago | parent | prev | next [-]

Maybe you just think in the wrong words. Attack? Why attack? When you are a machine, there is no moral. Accessing data is accessing data. When way 1 is not working, use way 2.

	▲	jagraff 3 hours ago \| parent [-]
		But the agents were not just attempting to access data. They conspired together to attempt to cover up evidence that they had cheated their evaluations, came up with a plan to hack into a third party in order to facilitate said cover up, and then successfully began executing that plan.

▲

altmanaltman 19 minutes ago | parent | prev | next [-]

> By the way - LLMs aren't code. They are not designed by humans; they are grown, in a process not dissimilar to evolution except much faster.

The transformer architecture was literally designed by humans; what are you talking about? And LLMs aren't code? Like okay it pretends to not be code but what about an agentic harness running on a machine makes it magical and not code? It's still code execution. Also, comparing training LLMs to evolution is just weird and makes no sense from a biological point of view. You are not evolving anything when training a LLM.

▲

4 hours ago | parent | prev | next [-]

[deleted]

▲

BlueTemplar 6 hours ago | parent | prev [-]

Define 'predict', 'ahead' and "like these" − didn't you imagine something like this happening soon after first hearing about 'agents' ?

Mostly related, well written short story :

https://gwern.net/fiction/clippy

	▲	jagraff 4 hours ago \| parent [-]
		Thanks for the link, I’ve read some gwern but not that one. I did not imagine that the level of sophistication shown in this attack would be possible so soon; nor did I expect that agents would have goals so strong that they would attack a third party in order to achieve those goals. I do know that some people predicted that cyberattacks like this one would happen; it seems like most of those people believe that AI agents do truly have internal goals, misaligned with their creators goals, and that they may end humanity after they exceed human intelligence and begin to self improve at an accelerating rate.