Just spent the last week or so porting TheRock to stagex in an effort to get ROCm built with a native musl/mimalloc toolchain and get it deterministic for high security/privacy workloads that cannot trust binaries only built with a single compiler.

It has been a bit of a nightmare and had to package like 30+ deps and their heavily customized LLVM, but got the runtime to build this morning finally.

Things are looking bright for high security workloads on AMD hardware due to them working fully in the open however much of a mess it may be.

I also attempted to package ROCM on musl. Specifically, packaging it for Alpine Linux.

It truly is a nightmare to build the whole thing. I got past the custom LLVM fork and a dozen other packages, but eventually decided it had been too much of a time sink.

I’m using llama.cpp with its vulkan support and it’s good enough for my uses. Vulkan so already there and just works. It’s probably on your host too, since so many other things rely on it anyway.

That said, I’d be curious to look at your build recipes. Maybe it can help power through the last bits of the Alpine port.

Wait ?

You don't trust Nvidia because the drivers are closed source ?

I think Nvidia's pledged to work on the open source drivers to bring them closer to the proprietary ones.

I'm hopping Intel can catch up , at 32GB of VRAM for around 1000$ it's very accessible

https://github.com/ROCm/TheRock/issues/3477 makes me quite sad for a variety of reasons. It shouldn't be like this. This work should be usable.