Maybe it's distro specific but I'm fairly sure mysql by default installed with skip networking around 2012, and with bind-address set to localhost since then. Also the root user is only configured from local by default but I'm not sure if that was true 14 years ago.

If the defaults are more secure than your examples, it's not fair to blame the database or the defaults.

And personally I hate it when software forces security requirements on me. Maybe I don't need an admin password. It's one reason I gave up on selfhosted gitlab - there was no option to reduce password complexity for my users, and those users were only connecting from the local network. The other reason being that it spammed 100GB of logs in a month and was using 11GB of RAM before I'd even gotten around to setting up the first repo.

I got bit by this with redis a couple years ago. The default install on my VPS's distro had it open to the world by default, and it even configured UFW for me (gee thanks!)

No password, owned within seconds of install. :/

[dead]

Or in one of my favorite coinages: "Moral Crumple Zones". "Just as the crumple zone in a car is designed to absorb the force of impact in a crash, the human in a highly complex and automated system may become simply a component—accidentally or intentionally—that bears the brunt of the moral and legal responsibilities when the overall system malfunctions." See https://ferd.ca/notes/paper-moral-crumple-zones.html

Yep and the car wasn't self-driving at the moment it crashed.

If a company advertised an LLM as perfectly safe and then it caused some damage, there would be a case against the company.

Have you used LLM tooling? It comes with warnings and explains that the user accepts the risk. Different levels of warning are supplied for the different levels of autonomy you can enable. The user has to understand the risk as they enable it.

This is not a new concept and it’s not an idea the LLM companies invented. It shouldn’t be surprising to anyone.

Constantly nagging the user is a liability model that also works well as a dark pattern for cases where you want the user to be less secure.

The system I'm comfortable with is to set the agent up as an unprivileged unix user, with no ability to change system configuration and no access to any files I didn't specifically give it access to. Need to let it access a file or a directory? chmod is your friend.

Second, it can pull from git, or submit a pull request, but not directly push. We have an existing system of code review for that, now also augmented by llms.

Thirdly, prevent it from sending anything but get requests to anywhere you don't want it to post stuff, with firewall configuration.

After that, turn the horrible security theater of it asking permission for anything off. So far we have had no incidents. It could of course still pull a malicious package from somewhere, that exfiltrates code using GET, but at least it can't send any credentials or user data over.

I suppose there would have to be a capability based model in conjunction with a user oversight model and a time model.

https://en.wikipedia.org/wiki/Capability-based_security

Thus some agents with higher capabilities can only be run with user oversight at the same time.

Some agents can not be run during some part of the day - for example these agents can not run within two hours of office closing time, and cannot run on weekends.

Maybe also the idea of agents writing code - throwing "out fully-fledged programs that you have to approve or reject permissions for."

Would work better with a capabilities based model where you choose capabilities for the program before hand, meaning the capabilities are not written by agent itself, you read through the code, some of it looks hairy but everything is fine, but oh no dumb human missed the part where agent writes to system32! But luckily enough the program you were expecting actually needed no write capabilities and thus when it tries to go past its assigned capabilities that part of the program fails and the exception is registered.

Googling it seems like lots of people have thought this (at least where Capability based security is concerned), which seems reasonable to me as it also seems pretty self-evident it must be this way. Have not really seen anything about time based controls but then that is probably because I'm not devoting a lot of effort as I am just doing a bit of procrastination to build up the energy to finish something off.

I think ultimately what it looks like it containing the blast radius if an agent does something bonkers.

The best case would be putting an agent in a VM and mounting the working directory there. Then you can allow it to run somewhat arbitrary actions while still being able to turn off the vm and restart it in a clean state.

The issue is, of course, that it doesn't fully prevent all possible problems an agent can cause. exfiltration is, IMO, basically impossible to stop. LLMs are exfiltration machines. The basic premise of all of them is "send us your code and a prompt and we'll do something good with it. But also if an agent decides run a command which installs a worm on a device on the network, you are hosed.

I don't think there's a way to make it secure while still permitting it unprompted external access

Eg: Any web request is a security vulnerability, there's no way to do it if the web requests are being made maliciously

Say that we have an agent with access to get requests, solely to a single site https://yoursite.com without subdomains. In this case multiple requests can be sent, and the time between requests can be used to exfiltrate personal data, similar to the coffee shop attack but without the subdomains. If the AI is able to make requests in any form, some information can be leaked, where the amount of leakable information is tied to information theory content of whatever side channel is being used. The only 0 information channel is.. never to make a request

You could also completely trust the 3rd party you're connecting to, but that to me seems like a hard error in the modern internet

I cannot help with your actual but this is giving me mild ptsd flashbacks to everyone on hn/slashdot constantly repeating how simple and perfect unix security is, just use user accounts!

As if the most valuable thing on my pc was running a program on the gpu or the printer as opposed to my email account.

Anthropic has gotten much better results by just having a different agent audit the actions of the original agent. It works surprisingly well

What a serious security model for a meatbag agent looks like? No, but seriously, an admin in a small org is a huge key-person risk in that they (or their stolen creds) can wipe enough and quick enough to effectively disable the business altogether.

More security conscious admins will at least segment their creds and implement four eyes principles somewhere, but were are back at square one of "asking user for confirmation".

Larger orgs, even if by necessity, segment their human agents, their creds and plaster four eyes principle liberally. But this relies on safeguards against agents colluding and ignoring some inputs, which sounds a bit scary for artificial agents.

Say you implement some swarm of agents, where access-enabled sub-agents are extremely restricted with system prompts and some access filtering. Then none of the agents in the swarm should be able to spawn themselves, otherwise a rogue agent can overwrite any safeguards. That, again, leaves the user with manually approving/denying network requests / hosts / sessions.

While I don't like anthropomorphising LLMs, the problem domain seems quite damn close to that of a key person going rogue within an org. The general solution seems to be liberal amounts of trust and ~~sweet compensation~~ gaslighting about replaceability.

For one, I’ve been working on a generic sandbox environment

github.com/brianv0/formwork

You should be easily able to hide/lock down files, network, and MCP tools from an agent and it shouldn’t be up to the agent.

[flagged]

[dead]

> Air Traffic Control is still primarily voice based, and simply up to the user on both sides to not make a mistake

The "user[s] on both sides" of ATC conversions have passed through the filters of rigorous training and certification. They also happen to communicate in a DSL designed to minimize misunderstandings, the DSL just happens to be based on English.

I do feel like that still needs to add a layer of interactivity to be complete.

From what I've seen most auto-approvers in coding harnesses either auto-approve or auto-reject, with no middle ground of escalating the decision to the user, and breaking down the pros and cons for the decision.

There's really no other alternative, though?

"Lets have the system that fails sometimes that we are trying to ensure does not fail check it self"

This is such an unserious approach.