| ▲ | masterj 6 days ago |
| Outside of the initial wave of security vulnerabilities and scrambling, it seems like the logical outcome of this over time is likely vastly more secure vm environments? |
|
| ▲ | pianopatrick 6 days ago | parent | next [-] |
| I think the main problem with that is the main problem with a lot of security tools. In order to do useful work, you need to provide a lot of tools and permissions. I.e. in theory the most secure might be a virtual machine with no network access. But then how do you access the LLM provider? Etc. |
| |
| ▲ | masterj 6 days ago | parent | next [-] | | I suspect capability models are going to get more popular https://en.wikipedia.org/wiki/Capability-based_security | | |
| ▲ | ch4s3 6 days ago | parent [-] | | I'm doing some work in this space[1], it's a really deep and hairy problem. There's a lot that you can do at the OS level sand boxing of course, but you may want some areas of you code to have network access and for some areas that use external dependencies to not have that access. Tracking where systems have side effects ends up being a lot of book keeping and a lot of languages that implement an object-capability model require you to thread caps through all of your calls, which IMO is bad ergonomics and a place where bugs creep in as functions accrue caps. Or sometimes they have rather superficial cap systems like Hack, or are rather awkward like in Deno. [1] https://march-lang.org/docs/capabilities | | |
| ▲ | thesz 6 days ago | parent | next [-] | | Why do you invent a new language for your work? Why did you not embed your language into another one, with type system that is superset of what you need? For example, there's capabilities expressed in Haskell: https://github.com/tweag/capability Capabilities there are tracked at type level and are subject to type erasure, if possible. | | |
| ▲ | ch4s3 5 days ago | parent | next [-] | | There are a few reasons for this. I really can't seem to wrap my head around Haskell in the wild as written by real people. I also wanted to tie capabilities into the build system tooling so that the build tool could verify caps and launch executables into an OS sandbox. Bolting on capabilities doesn't offer the same ability to enforce them, for example the March package manager ForgePM will reject packages that falsify their cap manifest. Moreover I was reading some papers that inspired the language, and wanted to try it. Cap(X) is also erased b the type checker during compilation in March. | |
| ▲ | tome 5 days ago | parent | prev [-] | | I'll blow my own trumpet and promote my own Haskell capabilities library, Bluefin: https://hackage.haskell.org/package/bluefin Bluefin is used in production, and as far as I know capability is not. | | |
| ▲ | ch4s3 5 days ago | parent [-] | | That looks really cool. Can you narrow Bluefin.IO to reads/writes separately? One of the things I've worked on is the ability to allow code to read files, even specific files, but deny writes. | | |
| ▲ | tome 5 days ago | parent [-] | | Yeah you can write a capability that encapsulates exactly whatever effects that you like! | | |
| ▲ | ch4s3 5 days ago | parent [-] | | Did this start as an effect system and then capabilities shook out naturally? | | |
| ▲ | tome 5 days ago | parent [-] | | Yes! It started as an implementation of the effect system I always wanted: effects passed on the value level, rather than implicitly on the type level. Once I'd done that I realise that it was actually a capability system (and that was the better way of describing it, because more people already know what a "capability system" is). | | |
| ▲ | ch4s3 4 days ago | parent [-] | | That’s not surprising there’s a lot of mechanical overlap between the two. It’s a really interesting relationship. | | |
| ▲ | tome 3 days ago | parent [-] | | Yeah, I think they're basically identical concepts. I'm writing an article on that topic right now. | | |
|
|
|
|
|
|
| |
| ▲ | masterj 6 days ago | parent | prev [-] | | That's a really interesting project! Being able to assert more, statically, about what our software is doing seems like a real growing need | | |
| ▲ | ch4s3 6 days ago | parent [-] | | Thank you, I really appreciate that. I'm really trying to make it easy to assert what code can do, then enforce it at compile time and optionally via the build tool at runtime via OS sandboxing integration. Supply chain attacks are also an area I'm exploring by having deps declare caps and then the build can scan and inspect binaries. It's been a real education. I talked to one of the people behind Caja and learned a lot. |
|
|
| |
| ▲ | WhyNotHugo 6 days ago | parent | prev | next [-] | | > But then how do you access the LLM provider? Etc. You can expose an HTTP proxy over a vsock into the VM. | | | |
| ▲ | tantalor 6 days ago | parent | prev | next [-] | | If "LLM provider" is part of the conversation, then you have already have ceded the security question. | |
| ▲ | bossyTeacher 6 days ago | parent | prev | next [-] | | It's the security vs convenience trade off. | |
| ▲ | 47635274172635 6 days ago | parent | prev [-] | | Invent new protocol just for LLM? | | |
|
|
| ▲ | Veserv 6 days ago | parent | prev | next [-] |
| That makes as much sense as saying that better gun technology results in body armor that can stop it. It might incentivize that, but in no way "results" in that; the fundamental technologys underpinning advancements in offense versus defense are fairly different. |
| |
| ▲ | matthewdgreen 6 days ago | parent | next [-] | | Maybe, but I think this misstates how security vulnerabilities work. Vulnerabilities are logic flaws, and we have every reason to believe that there is a maximum number of such flaws in any given system that allow exploitation; and even that we can conceivably develop logic that excludes any flaws. Whereas weapons and armor are devices that deliver and deflect/absorb energy, and any increase in the power of one means we need a corresponding increase in the other. Now maybe our understanding of logic systems is wrong, and it's just fundamentally impossible to develop programs that lack exploitable vulnerabilities -- that you can always "exploit with more energy". But there's no reason to believe the energy metaphor transfers to logic and intelligence. | |
| ▲ | dilyevsky 6 days ago | parent | prev | next [-] | | Very poor analogy - information security has very few commonalities with ballistics (duh) | |
| ▲ | ericd 6 days ago | parent | prev [-] | | Except in this case, the gun is the one directly improving the body armor. | | |
| ▲ | Veserv 6 days ago | parent [-] | | Only if you think penetrating holes in a paper vest and then patching those holes constitutes “improving”. If that worked Windows and Linux would be impenetrable fortresses with all the holes that keep getting punched in them. Cyberdemolitions expertise is about as relevant to cybersecurity as gun making is to bulletproof vest making. Necessary for validation, but not very related to the fundamental engineering and technology. |
|
|
|
| ▲ | ninininino 6 days ago | parent | prev | next [-] |
| We need better digital jailcells for our digital slaves basically. Or if you see AI as more tool and less entity, better gunsafes for our guns. |
| |
| ▲ | mcmcmc 6 days ago | parent | next [-] | | They are more comparable to a computer worm than anything else. Very strange (and disrespectful imo) to make the jump to slavery. A gun can’t be used at all inside it’s safe so I’m not sure that makes sense. A better metaphor would be making sure gun ranges have backstops capable of stopping contemporary payloads and sufficient range controls to keep people from shooting at cars on the highway. Outside of that you need registration requirements and gun control to make sure you can mitigate and track down perpetrators of gun crimes off the range. If they’re to be used in active conflict you need laws of war to govern the use of lethal force. If you use them to hunt, you need a hunter’s safety card and a current tag. | | |
| ▲ | Sha1rholder 6 days ago | parent [-] | | A gun cannot fire unless someone loads and triggers it, an LLM cannot operate computer unless someone translates what it said to shell scripts. An LLM is super comparable to a gun, not a computer worm which is consistently dangerous. | | |
| ▲ | mcmcmc 6 days ago | parent [-] | | Sure. I never said guns were a bad metaphor. I meant a worm was more comparable than an “entity” or “digital slave”. Agentic AI is putting the gun on a robot dog, taking the safety off and handing over fire control to an algorithm. If you extend it that far though guns are just any software. Or perhaps computers are guns and software programs are bullets, with LLMs manufacturing bullets from tokens. |
|
| |
| ▲ | helpfulclippy 6 days ago | parent | prev [-] | | maybe it's just information that really, REALLY wants to be free? |
|
|
| ▲ | cyanydeez 6 days ago | parent | prev [-] |
| This assumes your malefactors don't do malicious engineering, injects, social-agent engineering, etc. This same assumption is built around the singularity, the TAM of 30Trillion, etc. It's the idea that complexity will some how collapse upon itself in some bizarre borg like collective. Entropy is still going to win. |
