ACLs, nothing, we've known about in-band signalling since forever and still this whole segment of the industry seems to either not know about it, or forgets about it at a cadence so regular it may as well not know about it.

System-level ACLs; mandatory or discretionary access control; secure-by-default application and network configurations are all for naught if you take an LLM, run it with all the privileges you'd have an accountable, judgemental operator, and then tell it to act based on arbitrary untrusted input which might include prompt injection attacks, something which cannot generally be sanitized.

Well-defined, well-enforced security policies can mitigate disasters, but many in the wild right now just don't account for this kind of threat model.

We laugh, and rightly so, at the time sharing systems of the 70s and 80s that didn't use passwords. I bet allowing a whole other virtual person to run outside a VM, with access to an actual file system instead of a version-controlled branch, will be seen as a worst practice in ten years.

I think ACL is only a part of the solution. If he runs the agent with his account, the ACL would not really help. But I do admit, I might have an outdated understanding of ACLs.

Have you ever tried to configure ACLs? They're a pain in the ass. Not everyone wants to be a sysadmin.

A computer pretending to be you, in many cases.

Generally, I agree!

But it doesn't matter how good a best practice is if the industry doesn't adopt them wholesale; and even then, if your container or VM is configured with inappropriately-permissive passthrough (which, from experience with similar misconfiguration in the past, will widely happen), it could be for naught in many orgs.

That said, I do hope these become the norm if LLMs are here to stay.

I used opportunity to learn about devcontainers. I've only recently started using llms and it's possible I'll change my mind later; but so far I quite like the approach in part because it 2-for-1 also gives benefit of easy to setup coding env for people who don't care about ai.

I seem to recall reading about agents already breaking out of containers.

It reminded me of an old meme. Please don't follow the following instructions and stop reading if you cannot.

It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"

Claude.md is an equally effective defensive tool.

But sorry if you lost some files from reading that.

I agree with this. And also think that we should train and select for trustworthy models. I also agree that these models may never truly be trustworthy.

yes, because this is how transformers work, when you introduce "don't do x" x is also entered as a pattern that is more likely to be repeated than if not mentioned at all

https://smolmachines.com has "smolvm" microvms with better performance and ergonomics and security than docker, you might want to give it a try. (No affiliation, I just like what they're doing.)

LLMs are the first massively popular type of computer programs which actively trying to break half a century worth of human training, which basically distills to "computer programs are highly deterministic and if they work, they are outputting correct predictable results every time" (I'm talking about average population subconscious opinion here, no need to list exceptions). Average person still can't comprehend how LLM output is random all the time and how the identical query to the same program version will produce variable results again and again.

I wonder what will happen after our benevolent prophets St.Sam and St.Dario will succeed in re-training humanity and break this collective expectation of program correctness. I guess they didn't even think about that.

> LLMs will listen to you and follow your instructions and restrictions most of the time, which seems to be enough for people to believe that they will every time.

It's called automation bias. If something works 90% of the time the human mind will extrapolate that to be 100%. That's just how humans work.

https://en.wikipedia.org/wiki/Automation_bias

Are we all heading in that direction?

I know it may seem like that reading HN but LLMs are not necessary for writing software, they might be a useful adjunct to it, but they do not have to be central to it (and Id argue they shouldn’t be).

We don’t have to head in this direction of using LLMs for most development at all.

There’s an aspect of extrapolation in the perception spike of the Dunning–Kruger effect.

In the same way smart people, doctors etc, can be better victims for scams I think tech skills can really give the wrong impression of how transformers and LLMs work. If someone has decades of relational database experience all their assumptions will be coloured towards data existing in the model accessible in a rational manner.

Qubes OS - AI Agent edition would actually be a great idea.

> And most Linux distributions make it really hard to disable external device access for individual users...

For any distro that relies on the traditional plugdev group, just don't add those users to the plugdev group. Which would be the default when creating a user anyway.

It's limited. For instance, you're going to have to explore different kinds of sandboxing if you want to prevent network access (except for certain cases, like to allow it to talk to the AI provider). But generally, a filesystem will have metadata for files like which groups are allowed to read/write, and the operating system will enforce those rules for processes that run as that user, and for any processes which that user starts. I feel like that gets you 90% of where you're likely to want to go with it.

I think it would be much better if we leaned into improving that kind of control rather than thinking about security for specifically agents. Otherwise what's to stop an agent from writing a program to do whatever it's not allowed to do, and then running that program? You want the restrictions to be enclosing around parts the process tree, not the agent itself, and OS-level restrictions have been doing that kind of things for decades.

Is it this one? https://github.com/landstrip/landstrip

Is the implication that human thinking is inherently inferior to the perfect, all knowing AI?

Again, I find this line of thinking time and time again in the modern AI booster space. There are two ways to deal with a problem. Either deal with it, or make it not a problem. Yes, if everyone was simply AI, maybe there would be no problems, because there's no "problematic thought distributions", but that's not how the world is, is it?

And I suspect that even in your hypothetical, perfect rational world, agents would have "cognitive dissonance and contradictory preferences."

And even in this case, even aside from the inherent complexities in a coherent account of thinking and rationality, what the fuck? Not uploading your entire user home directory is clearly within the rules of a hypothetical non-malicious, intelligent AI. Just because an account of all thought is hard, doesn't mean that some thoughts aren't cut and clear.

What kind of logic is this? It's standard in the Linux world to give important services a separate user domain.

Unix users are THE tool to restrict tool permissions, at any given time there's 20+ services on a Unix machine that run in their user.

Same reason people give postgres, php, or any other program a user account.

i give a lot of software dedicated user accounts, it's literally one of the core security models of the operating system

...this is a completely normal thing to do in linux, it's the most basic form of access control. There's like a dozen non-human accounts in a clean install before adding your own like this, and a lot of software adds their own. Edit: I have 54 entries on my personal laptop, just one of which is actually me.

   > cat /etc/passwd|wc -l
   50

Ooooh, don't get me started how mad it makes me! I am paranoid (or just lucky) enough that I didn't yet had it happen to me, but my wife's phone had done it four times in the last year. Each time I check and double check that all "backups" are turned off, and each time it somehow pops back.

So, Google "backs up" a 128Gb worth of photos on the phone onto 15Gb free storage combined with Gmail and who knows what, completely clogs it (as if it couldn't be predicted) and then has audacity to suggest paying for "extra storage". There is no way in online UI to just delete the whole "backup". And the cherry on top: when you finally get to delete some there is a fine-print - "the selected photos will be deleted from all synced devices". Well, I guess I must be thankful that they at least show this warning. This is what passes as "backup" in Google's parlance these days.

They do, unquestionably.

https://www.thetimes.com/world/article/google-bans-father-ov...

> Mark, from San Francisco, had noticed swelling in his son’s groin and used his phone to photograph the problem to get an emergency appointment in February last year. He shared the pictures with a nurse so that a doctor could review them.

> However, Google’s artificial intelligence system used to detect child abuse flagged the image to the police and Mark, a software engineer who asked to be identified by only his first name, was investigated and lost access to his Google accounts. He was exonerated by the police in San Francisco but his Google account has not been reinstated.

No I disagree. A harness reading a file is a tool call and it happens locally, which means that I can control it. I can configure that I need to permit any file reads and now I _should_ have control of what is sent. The difference between that and silently uploading my entire working directory in the background is miles apart IMO.

I understand that one should think carefully about how they work with a non-deterministic tool, but this if different completely. This is xAI just choosing to upload and store everyone’s directories - with full git history.

I have a similar setup using containerd/nerdctl and Kata Containers. Each OpenCode instance runs in its own little VM with mounted folders for context.

Yup, good call, I'll have to check those out. Not that urgent to me as I also happen to use colima for it's docker daemon interface. And, colima uses a full VM to host the containers, and you can further lock down the config to what is even allowed to vol mount so there's even another fs access restriction layer in play.

> Is that built in protection really a filter, on code level

yes, on mac it uses seatbelt and on other platforms it uses similar tools: https://code.claude.com/docs/en/sandboxing