Remix.run Logo
mikestorrent 2 days ago

Yeah, it explains why I can't buy one of these. Now, what I am puzzled by is why none of the various PCIe (standard form factor) cards that offer m.2 nvme slots on them will work with any of my older computers; they have PCIe lanes to spare but I suppose the system firmware itself just doesn't know what to do with them.

Maybe I need to use a little sata SSD as /boot?

aidenn0 2 days ago | parent | next [-]

Author has an article on that too[1]

To explain what I learned on my own:

If you want to (for example) put 4 NVME drives (4 lanes each) in a 16x slot, then you need two things:

1. The 16x slot actually needs to have 16 lanes (on consumer motherboards there is only one slot like this, and on many the second 16x slot shares the lanes, so it will need to be empty)

2. You need to configure the PCIe controller to treat that slot as 4 separate slots (this is called PCIe bifurcation.

For recent Ryzen CPUs, the first 16x slot usually goes directly to the CPU, and the CPU supports bifurcation, so (assuming your BIOS allows enabling bifurcation; most recent ones do) all you need to do is figure out which PCIe lanes go to which slots (the motherboard manual will have this).

If you aren't going to use the integrated graphics, you'll need a 16x slot for your GPU. This will at best have 4 lanes, and on most motherboards (all 800 series chipset motherboards from all major manufacturers) will be multiplexed over the chipset, so now your GPU is sharing bandwidth with e.g. USB and Ethernet, which seems less than ideal (I've not benchmarked, maybe someone else has?).

In the event that you want to do a 4x NVME in a 16x slot I found that the "MSI X670E ACE" has a 4x slot that does not go throught the chipset, and so does the "ASRock B650M Pro x3D RS WiFi" either of those should work with a 9000 series Ryzen CPU.

ThreadRipper CPUs have like a gajillion PCIe lanes, so there shouldn't be any issues there.

I have also been told that there are some (expensive) PCIe cards that present as a single PCIe device to the host. I haven't tried them.

1: https://utcc.utoronto.ca/~cks/space/blog/tech/PCIeBifurcatio...

bpye 2 days ago | parent | prev | next [-]

Adapting a PCIe slot to a single M.2 slot should always work, so long as the SSD is actually using the PCIe lanes and not the SATA port that M.2 can also support.

Depending on how old the motherboard, or UEFI, is - it may not be able to directly boot from NVMe. Years ago I modified the UEFI on my Haswell-era board to add a DXE to support NVMe boot. You shouldn’t need to do that to see the NVMe device within your OS though.

As the other reply mentioned, if you want to run multiple SSDs on a cheap adapter your platform needs to support bifurcation, but if it doesn’t support bifurcation hope is not all lost. PCIe switches have become somewhat cheaper - you can find cards based on the PEX8747 for relatively little under names like PE3162-4IL. The caveat here is that you’re limited to PCIe 3.0, switches for 4.0+ are still very expensive.

adiabatichottub 2 days ago | parent | prev | next [-]

I have an older workstation with no NVMe sockets. You can find 4 socket NVMe PCIe cards that let you bifurcate a 16x PCIe socket into four 4x NVMe sockets. Great way to make a local flash ZFS pool.

https://www.ebay.com/sch/i.html?_nkw=Asus+Hyper+M.2

I still have to boot off a SATA SSD.

justsomehnguy 2 days ago | parent | prev [-]

> suppose the system firmware itself just doesn't know what to do with them.

Anything since socket 1155 can work and even Westemere/Nehalem should work too, except you don't really want a system that old.

I have a 4Tb M.2 drive in x1 slot in my GA-Z77X, works fine.

Not a boot drive and I didn't bother to look if the BIOS supports booting from NVMe, though a quick search says the support for that was started with Z97 chipset/socket 1150.

> use a little sata SSD as /boot

For Linux you can even use some USB thumbdrive, especially a small profile like Kingston DataTraveler Micro G2.

https://www.aorus.com/motherboards/ga-z77x-ud3h-rev-10/Key-F...