I'm also bemused by the number of people who think they've got an effective sandbox yet their sandboxed agent has access to all of their code, their github, and unrestricted web access.

▲

blcknight 2 hours ago | parent | next [-]

One bad npm package can really ruin your day. These things for me only run in their own VM with it's own GitHub account and basically nothing else

	▲	ofjcihen an hour ago \| parent [-]
		People probably think you’re being ridiculous but Shai Hulud had its very first attempt at manipulating AI lead analysis and I know of at least one company where that resulted in them getting pwned. This is only going to become more of a problem in the future and people need to educate themselves on the technical barriers to use because guardrails only sometimes work.

▲

Terr_ 2 hours ago | parent | prev | next [-]

I keep telling folks that they need to imagine LLMs (even "local" ones) as if you're farming it out to JS code running on some dude's browser somewhere: It can't keep a secret, and a determined person can make it emit anything they like.

We need to be asking what the most devious and malicious output could be, and whether what we do with that output (e.g. arguments to command-line tools) would still be safe.

	▲	NichoPaolucci an hour ago \| parent \| next [-]
		From my perspective, everyone is doing it. Security through obscurity - obviously if you’re harboring credit card numbers of users personal details, maybe take heed. But, if you’re a regular… run of the mill CRUD application, every other company is ALSO throwing caution to the wind. When hundreds of thousands of credentials are leaked into the funnel, does it really matter? I’m at a small company, and I try to push for security as much as I can, but the stakeholders truly do not care. They want to move fast. It’s just part of the new world I guess. If we get hit by attackers? I don’t know what happens. Sorry, we told you not to - you wanted to move quick and break stuff, this is how that culminates. I’m sure I’m not the only one.
	▲	skybrian 2 hours ago \| parent \| prev [-]
		We do have ways to avoid giving an LLM any secrets, but it needs to be the simple, default solution.

▲

norikaoda 2 hours ago | parent | prev [-]

[flagged]