Remix.run Logo
Filligree 2 hours ago

How many PCIe lanes are you allocating?

The card is obviously 16-lane, but it also has two ports; 40Gb total. In a server that’s fine, but if you want 10G in a desktop you’ll have a problem.

I’m probably not telling you anything new. NICs using newer PCI generations are rare as hen’s teeth. It should be possible to do this with four lanes, but isn’t…

Unless you find a 25G dual-port card, in which case the single lane my secondary slots hand out does at least suffice for 10G one way.

zamadatix 44 minutes ago | parent | next [-]

PCIe is given for a full duplex connection so 2x10G is still just 20G instead of 40G. For PCIe 2.0 x8 connection should get you full bandwidth on both ports simultaneously while x4 will fall just short for simultaneously usage (but still higher than 1 port).

Funnily enough, if you want a dirt cheap PCIe 3.0 based card the MCX353A-QCBT and MCX354A-QCBT give 1/2 ports of 40G QSFP+ and are dirt cheap. They support QSFP+ to SFP+ adapters, so you can plug a 10G SFP+ into the QSFP+ port, but they don't support 4x10G breakout unfortunately.

alexfoo an hour ago | parent | prev [-]

I'm only planning on using one of the SFP+ ports on each of the cards, the dual port cards were just more common and cheaper on eBay.

The specs say they require PCIe v2.1 x8 lane.

My Proxmox server is quite old and has a Gigabyte GA-X79-UP4 mobo and has loads of spare PCI slots. One slot is taken up by a generic graphics card as the Mobo has no on-board graphics. (I think I went for this mobo because of the number of SATA ports, but it was over 10 years ago so not entirely sure.)

My general Linux server is newer and has an ASUS Prime H610M-A D4 mobo. Only two PCI slots (not used at the moment) and so the Intel X540-DA2 will use up the PCIe 4.0 x16 slot leaving just a PCIe 3.0 x1 slot. But that's fine as this machine is just a CPU (i7-13700), 64GB RAM and a 2TB NVMe. Sticking a good graphics card in it for GPU related fun had been on my list for years but I never got around to it, now the prices are just insane so I'll ignore that for now or something second hand falls into my lap.