Hah, you really did the VM thing? A lot of people have suggested that to me but I didn't think it'd actually work. Pretty cool!

Yeah I'm pretty sure my onboard 10G Marvell AQtion ethernet is the source of most of my stability woes. About half the time any of these machines boot up, Windows bluescreens within the first couple minutes, and I think it has something to do with the iSCSI service crashing. Never had trouble in the old house where the machines had 1G network -- but load times were painful.

Luckily if the machines don't crash in the first couple minutes, then they settle down and work fine...

Yeah I could get higher-quality 10G cards and put them in all the machines but they seem expensive...

I've done a multi-seat gaming VM back in the day too. I don't think I'd want to do it again. Assigning hotplug USB devices was a pain: I mostly wanted unique USB devices per computer to easily figure which device was which. Though nowadays I would probably use a thin client Raspberry Pi running Moonlight to do it cheaply.

I think another issue is the limited amount of PCI-E lanes now that HEDT is dead. I picked up a 5930k for my build at the time for its 40 PCI-E lanes. But now consumer CPUs basically max out at 20-24 lanes.

Also with the best CPUs for gaming nowadays being AMD's X3D series because of its additional L3 cache, I wonder about the performance hit with 2 different VMs fighting for cache. Maybe the rumored 9950X3D will have 2 3D caches and you'd be able to pin the VMs to each CPU cores/cache. The 7950X3D had 3D cache only on half of its cores, so games generally performed better pinned to only those cores.

So with only 2-3 VMs/PC, and you still needing a GPU for each VM which are the most expensive part anyway, I'd pay a bit more to do it without VMs. The only way I'd be interested in multiseat VM gaming again would be if I could utilize GPU virtualization: split up a single GPU into many VMs. But like you say in the article that's usually been limited to enterprise hardware. And even then it'd be interesting only for the flexibility, being able to run 1 high-end GPU for when I'm not having a party.

Just buy used 10G hardware from an HFT firm :). Seriously, though, 10G gear is cheap these days.

I bet one could put an unreasonable amount of effort into convincing an Nvidia Bluefield card to pretend to be a disk well enough to get Windows to mount it. I imagine that AWS is doing something along those lines too, but with more cheap chips and less Nvidia markup…

There has got to be a way to convince Windows to do an overlay block device that involves magic words like “thin provisioning”. But two seconds of searching didn’t find it. Every self-respecting OS (Linux, FreeBSD, etc) has had this capability for decades, of course. Amusingly, AFAICT, major clouds also mostly lack this capability — performance of the obvious solution in AWS (boot everything off an AMI) is notoriously poorly performing.

It's been a couple years, but when I built our in-office render farm for my previous company I also got motherboards with built-in 10G because they needed 4GPU's and there simply no more PCIe slots left. There were so many connectivity issues, but eventually it was solved when we replaced the switches. When I first built the farm there was only one brand that sold cheap 10gbit ethernet switches, but a couple years later finally ubiquiti started making them as well and I think now all of the semi-pro brands sell 10gbit switches. Since we swapped to ubiquiti switches we had no more connectivity issues, not even with the cheap 10G interfaces.

The good intel 10G cards were not expensive at all by the way, I bought them for later additions, and they were cheaper than the premium we paid for the money-gamer motherboards that included 10G cards that I saw you were unhappy about too.

> Yeah I could get higher-quality 10G cards and put them in all the machines but they seem expensive...

Bulk buying is probably hard, but ex-enterprise Intel 10G on eBay tends to be pretty inexpensive. Dual spf+ x520 cards are regularly available for $10. Dual 10g-base-t x540 cards run a bit more, with more variance, $15-$25. No 2.5/5Gb support, but my 10g network equipment can't do those speeds either, so no big deal. These are almost all x8 cards, so you need a slot that can accomidate them, but x4 electrical should be fine (I've seen reports that some enterprise gear has trouble working properly in x1/x4 slots beyond bandwidth restrictions which shouldn't be a problem; if a dual port card needs x8 and you only have x4 and only use a single port, that should be fine)

I think all of mine can pxeboot, but sometimes you have to fiddle with the eeprom tools, and they might be legacy only, no uefi pxe, but that's fine for me.

And you usually have to be ok with running them with no brackets, cause they usually come with low profile brackets only.

I am not a gamer, but I found that https://moonlight-stream.org/ latency when streaming from my server to mbp is lower than that of my projector directly connected to said server. Might be easier to just get a beefy server with gpu passthrough than fight 10gbe drivers on 10 machines. Amd cards seem to work amazing for passthrough.

I'm building out a 10G LAN in my house (8k VR video files are ludicrously enormous) and while it's mostly Mac, where I use Thunderbolt to SFP fiber adapters, for my Windows PC I'm looking around at what PCI options to get, and haven't pulled the trigger.

If you make a decision on a 10G card (SFP or ethernet) I'd like to hear what you picked.

You can get used ones super cheap on ebay. The same applies to RAM, CPUs and other parts.

No need to buy new for most computing equipment unless you're looking for the absolute latest and greatest.

Yeah, gaming in a VM is fairly easy and reliable nowadays (the keyword to google for is VFIO). The cost savings is pretty substantial from consolidating multiple machines into one bigger machine. Unfortunately, there's an increasing number of games with anticheat that looks for being inside a VM.

> onboard 10G Marvell AQtion ethernet

I had similar problems with an Aquantia 10GbE NIC (which AQtion appears to be the rebranded name for, post-acquisition by Marvell), and it turned out to be the network chip overheating because it was poorly thermally bonded to a VRM heatsink that defaulted to turning on at something like 90C. Adding a thicker thermal pad and setting the VRM fan to always be on at 30% solved my problems.

> Hah, you really did the VM thing? A lot of people have suggested that to me but I didn't think it'd actually work. Pretty cool!

Another data point that it is indeed possible. I had a dual Xeon E5-2690 v2 setup with two RX 580 8GB cards passed through to separate VMs, and with memory and CPU pinning it was a surprisingly resilient setup. 150+ FPS in CSGO with decent 1% lows (like 120 if I remember correctly?) which was fine since I only had 60Hz monitors. I have a Threadripper workstation now, I should test out to see what kind of performance I can get out of that for VM gaming...

> Yeah I could get higher-quality 10G cards and put them in all the machines but they seem expensive...

I have had very good luck with Intel X540 cards. $20-40 on eBay, and there’s hundreds (if not thousands) available. They’re plug-and-play on any modern Linux, but need an Intel driver on windows if I remember correctly. I’ve never had one die and I’ve never experienced a crash or network dropout in the 9 years I’ve been running them. The Marvell chipset just seems terrible, unfortunately - I’ve had problems with it on multiple different cards and motherboards on every OS under the sun.