| ▲ | pmichaud a day ago |
| Nice work... but I still yearn for a stable ABI! |
|
| ▲ | JoshTriplett a day ago | parent | next [-] |
| The primary challenge in having a stable ABI is that we have generics. So, even if we never changed the calling conventions, an ABI is not as simple as exporting a symbol with the right name. We implement generics by providing MIR (our main intermediate representation) that gets compiled by the linking program for each instantiation of a generic. Swift handles this through the equivalent of Rust's `dyn`. We could have a stable ABI that's based entirely on `dyn`. Such an ABI would have some limitations, since there are traits `dyn` can't represent (anything that's generic over something other than the type that implements the trait, which is quite common), though we're working on some of those. Essentially, you wouldn't be able to have a `Vec<T>` in your ABI without either pinning down T or using something like `Vec<Box<dyn Trait>>`. Which is possible, but kind of awful. We're still planning to ship a stable ABI at some point, but that gives a flavor for some of the kinds of problems we'd need to solve in order for that stable ABI to be useful. (C++ typically handles generics by having types defined in header files, and those get compiled with the program consuming the library, which is effectively the same situation Rust is in. You could export symbols from a shared library for specific instantiation of a template, but people rarely do that.) |
| |
| ▲ | mamcx a day ago | parent | next [-] | | I wonder why not make the ABI contract explicit, instead of do what C do that froze the lang. Similar to : #[rust_abi]
struct MustConform
fn must_conform
So the user must write the shims instead of rust magically build it. This also mean that ABI changes become visible, and changes to rust not mean worries about ABI. | | | |
| ▲ | yrjrjjrjjtjjr a day ago | parent | prev | next [-] | | A generic could be seen as a pure (in the functional sense) program that is run at link time that takes various type information as input and outputs an implementation. | |
| ▲ | pjmlp a day ago | parent | prev | next [-] | | You are forgetting C++ has modules now, and extern templates, as implementations for common type parameters. | |
| ▲ | vaylian 19 hours ago | parent | prev | next [-] | | Would it be possible to provide a first version of the ABI that deliberately excludes support for generics? | |
| ▲ | IshKebab a day ago | parent | prev | next [-] | | I would be happy with something better than the C ABI even if it doesn't support everything in Rust on day one. | | |
| ▲ | barchar a day ago | parent | next [-] | | I think there's some work on that, getting at least slice and trait object ABIs that you can use in DSO interfaces. | |
| ▲ | JoshTriplett a day ago | parent | prev [-] | | Same. |
| |
| ▲ | gspr a day ago | parent | prev [-] | | Thank you for your work, it's always interesting to read your take. Forgive the naive question on something that's probably been discussed many times over, but could there be a middle ground when it comes to a stable ABI and generics? Something where the person building the library for dynamic linking chooses which explicit monomorphizations they'd like to provide for each public generic function (and type), and the compiler stamps out all combinations of those concrete instances? Could the way these are named and stamped out be standardized so that this could be treated as a stable ABI for non-generics? It seems to me that it would at least cover distros: they could make a pass over all their Rust code and determine which monomorphizations are needed throughout the distro. |
|
|
| ▲ | orlp a day ago | parent | prev | next [-] |
| We have it: unsafe extern "C" { }. An actual stable native Rust ABI sounds like a forward compatibility nightmare. |
| |
| ▲ | Xirdus a day ago | parent [-] | | An actual stable native Rust ABI is a prerequisite for distributing libraries as anything other than source code. | | |
| ▲ | ChadNauseam a day ago | parent | next [-] | | What is the advantage of doing so? Is it for cases where your source code is confidential but you want to sell access to your library? | | |
| ▲ | cogman10 a day ago | parent | next [-] | | Operating systems. If there's a CVE in tokio, the OS could theoretically publish a patched version of tokio and all apps that depend on that end up using the fixed code. As it stands now, the OS needs to publish updated versions of all rust apps that depend on tokio. | | |
| ▲ | afdbcreid a day ago | parent [-] | | That's not how it works basically anywhere except in C/C++ apps on Linux. | | |
| ▲ | cogman10 a day ago | parent | next [-] | | Yes and no. Yes, in that it's primarily a linux thing as MS has gone the route of saving off basically all dlls for all their OSes for backwards compatibility (though they still backpatch). No, in that other than Rust and Go (and maybe a few others that emit single binaries, zig?) this is how it works with OS distributed programming language packages. Python, ruby, java, perl, C#, the OS will for all of them control distribute and update the libs independent of the apps that use them. Also, C/C++ still make up a huge percentage of all apps in pretty much any OS. | | |
| ▲ | the8472 a day ago | parent | next [-] | | > prerequisite for distributing libraries as anything other than source code. > Python, ruby, java, perl, C#, the OS will for all of them control distribute and update the libs independent of the apps that use them. Those are all JIT'ed languages, so they're distributed in source or intermediate form, not as fixed-ABI-shared-libraries. It's really just C where shared libs really work. Even C++ has wrinkles. | | |
| ▲ | cogman10 14 hours ago | parent [-] | | > so they're distributed in source or intermediate form, not as fixed-ABI-shared-libraries. I agree. These languages all have the benefit of what an ABI provides due to their nature. Go and Rust are outliers because they don't send out intermediate versions. My point was more that the effect of being able to hot-patch applications on a system is what makes an ABI desirable, not that an ABI is necessary for that. For a language like rust, it's pretty necessary. Though it would be interesting if you could send out an intermediate compiled form of rust and do a JIT on that. WASM is pretty close to doing that, but rust doesn't support dynamic linking of modules (AFAIK). > Even C++ has wrinkles. Yeah, I somewhat hesitate to say C/C++ as being ABI stable as it's really just C and C++ uses that. Once you use any C++ features you quickly get into realms where things are ill defined. You end up needing gross things like COM in order to properly use C++ shared libs. |
| |
| ▲ | afdbcreid a day ago | parent | prev [-] | | > No, in that other than Rust and Go (and maybe a few others that emit single binaries, zig?) this is how it works with OS distributed programming language packages. Uhmm... No? E.g.: - Python has long strongly recommended virtual environments. - Not to talk about JavaScript which of course manage things per project by default. - I don't remember what was the state of things in .NET Framework, but in modern .NET NuGet packages are distributed as DLLs with your app. I don't know other ecosystems well but I believe they aren't different. | | |
| ▲ | cogman10 a day ago | parent [-] | | > Uhmm... No? Ummm... yes. How software is maintained by an OS is different from how software is maintained by 3rd party developers. - Python has long strongly recommended virtual environments. Yes, if you are developing a private application. But if that application gets adopted by the OS then you can expect all it's dependencies to be sent to a place like `/usr/lib/python3/dist-packages`. > Not to talk about JavaScript which of course manage things per project by default. I didn't bring it up because it's one I'm not familiar with. I expect OS maintainers will do similar things with it that they do with python but I'm not 100% on that. > I don't remember what was the state of things in .NET Framework, but in modern .NET NuGet packages are distributed as DLLs with your app. And if your .net application ends up installed as an OS package you can expect those dlls to be stored in /usr/lib. If you are doing C/C++, as a 3rd party app, you aren't putting the .so files or the .dll files in /usr/lib or C:\Windows\System32. You are storing them in the same directory as your application. But if the OS decides to distribute your app, you can expect your .so files to get put into /usr/lib and even System32 (very very rare for MS to do that, but it does happen when, for example, we are talking about drivers). Edit: Re: node. Here's the debian guidelines for how dependencies ultimately get shredded and distributed. https://wiki.debian.org/Javascript/Nodejs/Manual?pow_referer... | | |
| ▲ | afdbcreid a day ago | parent [-] | | But that's the thing: almost nobody ships software via the OS today. Even on Linux software is often managed in other ways. | | |
| ▲ | cogman10 a day ago | parent [-] | | Let me rewind this to the top of the thread. > What is the advantage of doing so? To which I responded > Operating systems. To which you said > That's not how it works basically anywhere I think we have reached the point in the thread where you concede that yes, there is somewhere where it's how it works and yes it does apply to languages that aren't C/C++. So now you're shifting the goalposts and topic to > almost nobody ships software via the OS today. That was never something I was trying to debate. The topic was "why would you want ABI stability". And the answer is that if I wanted to rewrite coreutils in rust, then it'd be really useful to have synchronized crates with a stable ABI so I could potentially update those crates independent of the apps that depend on them. | | |
| ▲ | afdbcreid a day ago | parent [-] | | I agree that sometimes other languages are packaged this way as well, and maybe I was moving the goalposts. My original meaning was that only for C/C++ (on Linux) the recommended method of distribution is via the OS, while other languages have their own mediums (not requiring stable ABIs). So when they are packaged like that that's mostly because the OS itself or its services need them. |
|
|
|
|
| |
| ▲ | gspr a day ago | parent | prev | next [-] | | Well yes exactly, because essentially nothing else has a stable ABI. | |
| ▲ | Xirdus a day ago | parent | prev [-] | | Because only C/C++ apps have stable ABI. Everyone else would love to do it too but they're S.O.L. | | |
| ▲ | afdbcreid a day ago | parent [-] | | That's not true. Scripting languages of course need no ABI. Managed languages like C# and Java often keep compatibility of the bytecode. Only AOT languages like Go and Rust cannot. |
|
|
| |
| ▲ | xigoi a day ago | parent | prev | next [-] | | Interoperating witt other languages without having to use C as a middleman? | | |
| ▲ | saghm a day ago | parent | next [-] | | I agree that using the C ABI for interop between arbitrary non-C languages feels weird, but it's not like that wasn't a thing long before Rust ever existed. If you truly want a modern ABI for languages that are not C, something like the WASM component model is probably the closest thing that exists today. | |
| ▲ | Xirdus a day ago | parent | prev [-] | | An unstable ABI is entirely sufficient for that, as long as you update both sides in tandem. |
| |
| ▲ | flykespice a day ago | parent | prev | next [-] | | Yep, and it also imposes a big security risk relying on a closed binary blob that could be injected anytime with a backdoor. Afaik, odin seems to get it right requiring libraries to be distributed source only. | |
| ▲ | xtajv a day ago | parent | prev [-] | | There is another thread complaining about compile times and tmp dir sizes. Stable ABI dramatically improves the problem space there (without necessarily having anything to do with business model). |
| |
| ▲ | SkiFire13 a day ago | parent | prev | next [-] | | It's not a strict prerequisite, you can just require the library being distributed to use extern "C". | | |
| ▲ | gspr a day ago | parent | next [-] | | Well yes, but then your interface is a C ABI. That's fantastic if the side using my library is any language where C is the only common ground. When the application side is Rust though, it leaves a lot to be desired. It would be delicious to actually interface in a way that respects Rust! Hell, even if all exposed functions and types would need to be fully monomorphized at the interface, it would be a lot nicer to pass a `&'a [i64]` rather than a `const * int64_t` together with a `size_t n` and have to marshall the slice on both sides. | |
| ▲ | Xirdus a day ago | parent | prev [-] | | Or to put it another way, it is a strict prerequisite and Rust already meets it via extern "C". |
| |
| ▲ | wahern a day ago | parent | prev | next [-] | | Another year or three of insane memory and disk prices and people will come around. A stable Rust ABI would be nice for not just Rust, especially if they refine or extend (formally or de facto) the SysV ABI to optimize passing and returning of discriminants. That could then smooth the way for developing these features in C and C++, such as a primitive error-result type that saw some brief attention in both C and C++ WGs a couple of years ago. Doing it properly--absolute minimum cost, ABI stable, and cross-language--seemed to require too much orchestration across the ecosystem so enthusiasm and ambition seemed to peter out. The work required for a stable Rust ABI could provide the inertia to get that done. | |
| ▲ | afdbcreid a day ago | parent | prev | next [-] | | Rust will almost certainly never have a stable ABI by default, and libraries will continue to be compiled from source by default. It might one day have an opt-in stable ABI. | |
| ▲ | fallingbananna a day ago | parent | prev | next [-] | | If you can drop the "native" requirement, then Wasm is also a way. | |
| ▲ | redox99 a day ago | parent | prev [-] | | Why would you want libraries in any other format? | | |
|
|
|
| ▲ | iknowstuff a day ago | parent | prev | next [-] |
| Follow or contribute to efforts on a minimal crABI: https://github.com/joshtriplett/rfcs/blob/crabi-v1/text/3470... |
| |
|
| ▲ | danhor a day ago | parent | prev | next [-] |
| I don't think a stable ABI has enough advantages to be worth it. C++, IMHO, is a great examples of the downsides of a stable-enough ABI, where stability was never guaranteed, but is now so entrenched that it'll likely never be broken again (limiting the language & the usability in a bunch of ways). It might make sense to have a subset Swift-style stable ABI, but that's a huge engineering effort, has lots of overhead and is also likely not worth it IMHO. |
|
| ▲ | stillpointlab a day ago | parent | prev | next [-] |
| stable ABI sounds great, but anything/everything that reduces build times is what I really want |
| |
|
| ▲ | Svoka a day ago | parent | prev [-] |
| Why the yearning for prebuilt Rust libraries? Seems like a nightmare which does not need to happen. |
| |
| ▲ | gspr a day ago | parent | next [-] | | Dynamic linking is useful. Not to everyone and not always, but it is. | | |
| ▲ | kibwen a day ago | parent [-] | | Rust supports dynamic linking. What a stable ABI provides is cross-toolchain dynamic linking. |
| |
| ▲ | Woshiwuja a day ago | parent | prev | next [-] | | [dead] | |
| ▲ | silon42 a day ago | parent | prev [-] | | So, all Linux will be Gentoo? | | |
| ▲ | Svoka a day ago | parent [-] | | what? There is stable ABI. It's C's stable ABI. If you are reimplementing existing libraries like most of rust rewrites are, you already have ABI you have to follow. |
|
|
