Has anyone thought about making the linux kernel roughly compatible with NT? Like how FreeBSD is compatible with Linux? I know it'd definitely be harder as NT is proprietary but syscalls (in my very uninformed opinion) seem all that difficult to implement, even without a userland

At what level do you mean that? Kernel level? Driver level?

Wine[1] is the de facto compatibility layer with NT executables. Driver compatibility is too complex and obscure to worth the while. Often information is undocumented or hard to get.

There are a few implementations of windows behaviors at kernel level for a few subsystems features, ntsync, samba, ntfs, etc. they can be used by wine to improve compatibility or performance

[1]: https://www.winehq.org/

FreeBSD is not "compatible with Linux", it provides a way to run Linux applications under a Linux-like syscall environment. What you're suggesting is as if you could load Linux kernel modules into the FreeBSD kernel.

The issue with NT is the driver ecosystem. You'd have to reimplement a lot of under-documented NT behavior for NT drivers to behave themselves, and making that work within the Linux kernel would require major architectural changes. The userland is also where a lot of magic happens for application compatibility.

It's far from "roughly compatible with NT", but the Linux kernel does accept changes to make supporting Windows applications more efficient.

example: ntsync

Me as a kid thought this would be a great idea, and started implementing a PE binfmt. I actually did make a rudimentary PE binfmt, though it started to occur to me how different Windows and Linux really were as I progressed.

For example, with ELF/UNIX, the basic ELF binfmt is barely any more complex than what you'd probably expect the a.out binfmt to be: it maps sections into memory and then executes. Dynamic linking isn't implemented; instead, similar to the interpreter of a shell script, an ELF binary can have an interpreter (PT_INTERP) which is loaded in lieu of the actual binary. This way, the PT_INTERP can be set to the well-known path of the dynamic linker of your libc, which itself is a static ELF binary. It is executed with the appropriate arguments loaded onto the stack and the dynamic linker starts loading the actual binary and its dependencies.

Windows is totally different here. I mean, as far as I know, the dynamic linker is still in userland, known as the Windows Loader. However, the barrier between the userland and kernel land is not stable for Windows NT. Syscall numbers can change during major updates. And, sometimes, implementation details are split between the kernel and userland. Now, in order to be able to contribute to Wine and other projects, I've had to be very careful how I discover how Windows internals works, often by reading other's writings and doing careful black box analysis (for some of this I have work I can show to show how I figured it out.) But for example, the PEB/TIB structures that store information about processes/threads seems to be something that both the userland and kernel components both read and modify. For dynamic linking in particular, there are some linked lists in the PEB that store the modules loaded into the process, and I believe these are used by both the Windows loader and the kernel in some cases.

The Windows NT kernel also just takes on a lot more responsibilities. For example, input. I can literally identify some of the syscalls that go into input handling and observe how they change behavior depending on the last result of PeekMessage. The kernel also appears to be the part of the system that handles event coalescing and priority. It's nothing absurd (the Wine project has already figured out how a lot of this works) but it is a Huge difference from Linux where there's no concept of "messages" and probably shouldn't be.

So the equivalent of the Windows NT kernel services would often be more appropriate to put in userland on Linux anyways, and Wine already does that.

It would still be interesting to attempt to get a Windows XP userland to boot directly on a Linux kernel, but I don't think you'd ever end up with anything that could ever be upstreamed :)

Maybe we should do the PE binfmt though. I am no longer a fan of ELF with it's symbol conflicts and whatnot. Let's make Linux PE-based so we can finally get icons for binaries without needing to append a filesystem to the end of it :)

That would require (among many other things) a stable driver API -- one of the things NT gets right and linux is wrong on. Linus has been quite clear that he does not see things this way. So ... not going to happen