> You can run the DLL in a "shim" subprocess that proxies function calls over IPC.

That doesn't address the need of some DLLs to malloc() resources in the context of the applications linking to them.

This problem _cannot_ be solved _generically_. Any solutions are extremely API-specific and impose restrictions on their users (the linking applications) which, if violated, will lead to Undefined Behavior.

Edit: as an example of cases which must bind resources in the application's context: see the Classloading in C++ paper at <https://wanderinghorse.net/computing/papers/index.html#class...> (disclosure: i wrote that article).

▲

LegionMammal978 3 days ago | parent | next [-]

"Extremely" API-specific is a bit much. One way to do it would be to guard all DLL-implemented functions, and all access to DLL-associated resources, with a reference-counted token, or otherwise use a language-level mechanism (such as Rust's lifetimes) to ensure that all resources are dead by the time of closure. This would take some tedious wrapper-writing, but it's not so complex that it couldn't be done by a source generator or whatever.

Of course, C/C++ applications written in the traditional model with static data everwhere would have difficulty not leaking tokens and holding the DLL open, but it's still far from impossible to write such a safe API.

> That doesn't address the need of some DLLs to malloc() resources in the context of the applications linking to them.

If there is a context boundary and really such a need, then the DLL can keep a list of all such resources, and destroy all those resources once closed. Access to them would similarly have to be protected by a token.

▲

zbentley 3 days ago | parent [-]

> One way to do it would be to guard all DLL-implemented functions, and all access to DLL-associated resources, with a reference-counted token, or otherwise use a language-level mechanism (such as Rust's lifetimes) to ensure that all resources are dead by the time of closure.

That's true, but those approaches are only viable if you trust the DLL in question. External libraries are fundamentally opaque/could contain anything, and if you're in a tinfoil-hat mood, it's quite easy to make new libraries that emulate the ABI of the intended library but do different (maybe malicious, maybe just LD_PRELOAD tricksy) things.

Consider: an evil wrapper library could put the thinnest possible shim around the "real" version of the library and just not properly account for resources, exposing library (un)loaders to use-after-free without much work, even if the library loaders relied upon the approaches proposed.

Since there aren't good cross-platform and race-condition-free ways of saying "authenticate this external library via checksum/codesigning, then load it", there are some situations where the proposed approaches aren't good enough.

Sure, most situations probably don't need that paranoia level (or control the code/provenance of the target library implicitly). But the number of situations where that security risk does come up is larger than you'd think, especially given automatic look-up-library-by-name-via-LD_LIBRARY_PATH-ish behavior.

▲

LegionMammal978 3 days ago | parent | next [-]

If you load malicious code into your address space and execute it, then it can always do malicious things to your data. If you load malicious code into a separate process and execute it, then it can almost certainly do malicious things to your data, unless you put it into a locked-down user context and trust your OS and environment not to have any local privilege escalations (lol). The only real way to load untrusted native code is to put it in an OS-level container and communicate via IPC, or better yet, put it in a VM and communicate via a virtual network.

The measures I suggested before were all in the context of buggy users that can't resist the urge to keep references to the library's resources lying around all over the place. But untrusted code can never be made safe with anything short of a strong sandbox.

▲

johnisgood 3 days ago | parent | prev [-]

> Since there aren't good cross-platform and race-condition-free ways of saying "authenticate this external library via checksum/codesigning, then load it", there are some situations where the proposed approaches aren't good enough.

Sign your libraries with Ed25519 and embed the public key in your app, verify before load. How is this not cross-platform enough?

Of course you still introduce a TOCTOU (time of check, time of use) race condition, which is why oftentimes you want to first check, load, then check again.

A common solution, however, is opening the library file once, then verify checksum/signature against trusted key, and if valid, create a private, unlinked temporary file (O_TMPFILE on Linux), write the verified contents into this temporary file, rewind and dlopen() (or LoadLibrary()) this temporary copy. Because the file is unlinked after creation (or opened with O_TMPFILE), no one else can swap it out, and you eliminate TOCTOU this way because you only ever read and load the exact bytes you verified. This is how container runtimes and some plugin systems avoid races. BTW on Linux you can use memfd_create() which creates an anonymous, in-memory file descriptor. You can do the same on Windows and macOS. Then you can verify the library's signature / hash, copy verified contents into a memfd (Linux) or FileMapping (Windows), and then load directly from that memory-backed handle.

TL;DR: never load from a mutable path after verification. Verifying untrusted binary bytes into a sealed memfd, for example, is race-safe.

FWIW, for applications I use firejail (not bubblewrap) for all applications such as my browser, Discord, LibreOffice, mupdf, etc. I recommend everyone to do the same. No way in hell I will give my browser access to files it does not need access to. It only has access to what it needs (related to pulseaudio, Downloads directory, etc), and say, no way I will give Discord access to my downloaded files (or my browser history) or anything really, apart from a directory where I put files I want to send.

	▲	3 days ago \| parent [-]
		[deleted]

▲

10000truths 3 days ago | parent | prev [-]

True, it's application specific. The typical reason an application would need to load/unload DLLs cleanly and repeatedly (where the caveats of dlclose are in full force) is for a plugin system with hot-reload functionality. And for that use case, the expected API/ABI is known in advance, so the shim can be tailored for it.