At least on Linux, you're guaranteed to get the top 16 bits free on at least x86[1] and ARM[2]. Maybe other OSes are less nice, but generally an OS doesn't really have a reason to force down the full 57-or-whatever-bit range before the program actually requests 256 terrabytes of virtual address space.

It is generally kinda sad though that there's not a way to request from mmap or equivalent that the result is in a specific range of memory (in (0; 1<<48) here). Would be useful for JIT-compiling code that needs to call into precompiled functions too.

[1]: https://www.kernel.org/doc/html/v5.8/x86/x86_64/5level-pagin...

[2]: https://www.kernel.org/doc/html/v5.8/arm64/memory.html#bit-u...

▲

bigstrat2003 a day ago | parent [-]

> Maybe other OSes are less nice, but generally an OS doesn't really have a reason to force down the full 57-or-whatever-bit range before the program actually requests 256 terrabytes of virtual address space.

The fact that Linux does this isn't nice, it's a huge mistake. It means that the kernel can't automatically use 5-level page tables on processors that support it, because backwards compatibility guarantees mean the programs must be able to use those bits in a pointer. AMD was smart enough to throw an exception if programs use those bits in a pointer (thus guaranteeing forward compatibility), so why Linux didn't follow suit is puzzling.

	▲	dzaima a day ago \| parent [-]
		Indeed most architectures error if the top bits are non-zero, but that's trivially worked around by just a manual `x & 0xffffffffffff` or `x << 16 >> 16` or similar, and software already utilizes that (and indeed must do that to not immediately break on x86). It is somewhat unfortunate to just force the larger address space to specific mmap usage, but it's hard for me to imagine that many programs actually needing more than 256TB of virtual memory that aren't doing so in a very-specialized way. Certainly much less frequent than the already-infrequent (but very much existing, and significant! incl. both Firefox/SpiderMonkey and WebKit/JavaScriptCore) cases of programs utilizing top 16 bits being zeroes. Then there's the option of mmap returning ranges from from the low 2^48 while possible, and using larger addresses only when that completely runs out; should mean existing software works fine before it needs more than 256TB of RAM, and, if the software checks the top bits of mmap's result being zeroes, it's not negatively affected anyway. Really the proper solution is to go back in time and make mmap have separate lower and upper bounds addresses though.