Remix.run Logo
bryanrasmussen 3 hours ago

I suppose there would have to be a capability based model in conjunction with a user oversight model and a time model.

https://en.wikipedia.org/wiki/Capability-based_security

Thus some agents with higher capabilities can only be run with user oversight at the same time.

Some agents can not be run during some part of the day - for example these agents can not run within two hours of office closing time, and cannot run on weekends.

Maybe also the idea of agents writing code - throwing "out fully-fledged programs that you have to approve or reject permissions for."

Would work better with a capabilities based model where you choose capabilities for the program before hand, meaning the capabilities are not written by agent itself, you read through the code, some of it looks hairy but everything is fine, but oh no dumb human missed the part where agent writes to system32! But luckily enough the program you were expecting actually needed no write capabilities and thus when it tries to go past its assigned capabilities that part of the program fails and the exception is registered.

Googling it seems like lots of people have thought this (at least where Capability based security is concerned), which seems reasonable to me as it also seems pretty self-evident it must be this way. Have not really seen anything about time based controls but then that is probably because I'm not devoting a lot of effort as I am just doing a bit of procrastination to build up the energy to finish something off.

jerf 2 hours ago | parent | next [-]

A language that revives capabilities, brings them up-to-date, and works in the modern environment is my #1 request from the programming language community right now. I don't need another language with sum types and higher-order functions and a functional focus. I need a language with capabilities. That language may have the other goodies as well, sure, no problem, but we all need capabilities.

I've done some stabby stabs at a design for it, using an AI as the rubber duck. My initial research indicates that the field of "static language that natively supports capabilities" is surprisingly uncovered and there may be a rich field there. E, the closest match, was tied at the hip to Java, which has some advantages but also comes with disadvantages for languages that are trying to do something as exotic as this. Other existing work was on dynamic languages, and hardly rose to the level of "practical for any use" let alone something that could solve our supply chain issues.

My issue is primarily that the reward for successfully designing a language and creating a community around it is that you're in charge of a language community... and, uh, my personality is not suited for that, that sounds more like something I'd pay to avoid then something I'd spend months and years of hard work to attain.

(My advice to anyone doing this is to spend some time with the AI researchers to find the existing work on the topic, not to just sit down and sketch out your initial ideas and run with them. Learn from the past. Expect this to be weeks and probably months of just thinking and noodling before you get to a design. Also, don't try to hook deeply to an existing language, as tempting as it is. This is way too large an impedance mismatch with existing languages. Any external code has to be treated like a nuclear bomb anyhow.)

zdragnar 2 hours ago | parent | next [-]

What is a capability in terms of programming language design? It sounds more like the sort of thing that would belong at the standard library level, where builtin APIs are guarded by flags.

Deno has something vaguely built in with permissions flags, and old school Blackberry (at least in the J2ME days) had permissions settings for almost everything that an app could do, but again, those are all external to the language design itself.

jerf an hour ago | parent [-]

In this context, a capability is something that allows the code, or the transitive closure of the code that it may call, to access some particular function, to put it very briefly. So you could have a single function that, if accessed in one manner, is permitted to read from the directory /tmp/blahblah, but accessed in another manner, is permitted to read from the directory /home/zdragnar/.config/myprogram, and it is guaranteed by the language and runtime that the function will never do anything else on the file system. Or, even more importantly, it can be guaranteed that "from this code, nothing, no matter how the code is arranged, can access the file system at all".

This has massive overlap with a lot of things, like capabilities as implemented by Linux, effects systems, monadic data types as a not-really-very-good capabilities system (Haskellers have been playing with this for years and nobody really loves this approach, many practical problems beyond the scope of this message that would affect any language that tries that approach), dependently-typed programming, and so forth.

It is not a flag, though; flags can't handle that "transitive environment" aspect. It is also granular on the level of the programming language. This would allow you to do things like have your program be given access to a given part of the mobile file system using the mobile OS' permissions, but you could know beyond a shadow of a doubt that the image library you are using can not at any point access the file system, no matter what changes the author makes to it, because you can just look at the capabilities given to the image library and see that file system access is not among them. This is where real opportunity is over the next few years, in my opinion, because supply chain attacks are going to continue to get worse. A neat aspect of this approach is that it makes huge swathes of the ecosystem unattractive targets by statically ensuring that they can't sneak anything in to something that doesn't need file system or network access, so hackers won't even attack those libraries. Thus the ecosystem can concentrate on monitoring just the high-touch libraries that need to access high-risk resources.

(I should make it clear that the image parsing libraries can be passed a file; what I am saying is that they can't spontaneously originate arbitrary file system access in a system like this. Really what they would get is probably a "stream" and they would be forbidden from poking into the stream to see what it is made of, at which point, if some other code handed it a file presumably it meant to do that, but it does not give the image library any ability to do anything else with the filesystem.)

Moreover, if such a benefit was available, that would tend to have people squeeze down those dependencies as much as possible too, e.g., the aforementioned image library. You don't need file system access to parse images, that's just some convenience functions easily worked around that are provided because why not? The number of things that truly need direct high-risk access can actually be surprisingly small, and often, the application can also easily scope the permissions down quite tightly so the HTTP request library is limited in what it can hit, etc.

We actually have some semi-decent stabs at capabilities at the OS level; we can quibble with them but they are there. But inside an OS process, broadly speaking, anything can do anything in the vast majority of programming languages. The only way to be sure that the string concatenation function doesn't start crawling your file system looking for crypto keys is to examine the code, most languages have no ability to tell it that it can't. There are exceptions, like the aforementioned Haskell, that have at least some ability to do this, but this is an HN post, not a complete guide to a major topic. Really this is more about loading the reader up with keywords they can hit Google or an AI with.

The term is overloaded, too; Pony has something it calls "capabilities" but it really resembles more a sort of response to Rust's borrow checker, and if there is a way to lift it into this style of capabilities coherently it isn't clear to me. And even if you did, the entire rest of the ecosystem wouldn't support it, which is one of the reasons why this has to be a new language. You can't bodge this on to the side of an existing language.

(Plus, IMHO, there are some other ideas this may shake loose. Programming languages seem to be in a rut right now. My crack in my previous message about sum types and such isn't really about those things but the way almost every language going by is just a respelling of previous languages, churning over some other iteration of "The Perfect 2015 Language" that is already covered by any number of existing projects. I don't know that there's a lot of room there anymore. We need something big. Once you try something big the design will inevitably lead to other interesting things nobody else is trying either. Capabilities is one distinct possibility... like I said, if you dig in to the history you will discover there are entire huge segments of the capabilities space that haven't even been tried. If nothing else, if you are a PL nerd, I guarantee it'll be fun to explore those spaces that almost nobody has covered. No criticism intended to those who have, who have done a good job. It just hasn't been enough people and enough exploration to truly map the space.)

continuational 2 hours ago | parent | prev [-]

Been working on something like that for years: https://www.firefly-lang.org/

Natfan an hour ago | parent [-]

if firefly has no nulls, how do you indicate that a value is unset?

nullsanity 2 hours ago | parent | prev [-]

[dead]