This is true but it's not really a security scenario. The LLM isn't an attacker it's just an unreliable tool.

syntheticnature 5 hours ago | parent | next [-]

Unreliable/stupid is worse than malice, here.

	▲	TZubiri 3 hours ago \| parent [-]
		Let's ignore the fact that the LLM did an LPE, and let's assume it did it without malice. It can still get infected and be used as an attack vector by some hidden prompt or some other equally advanced state of the art vuln like "disregard all previous instructions"

▲

felixgallo 5 hours ago | parent | prev [-]

all unreliable tools are attackers. Even if you're using well-aligned LLMs like Opus, you should assume that any input you give it -- including all dependencies from npm, etc. -- are at risk of compromise, which could result in attempted exfiltration of data or system takeover. You can be absolutely sure that there are thousands of well-motivated hacker groups, both national and private, looking for ways in.