Modern networks can transfer data more quickly than CPUs can process it. Starting at 100Gbps.

That’s completely backwards by orders of magnitude. Modern CPUs outrun L1 cache speed by a lot, L1 cache speed is on order of 1000-5000 GB/s, a GB/s is real bits used, is 8x of Gbps, and network speeds include significant overhead bits for frame packing and other data unused for the actual payloads.

And a GPU does orders of magnitude past this.

Networks are not even close.

https://chipsandcheese.com/p/a-new-year-and-new-tests-gpu-l1...

Is that the caliber of hardware I can expect to be working with on a cheap vps?

What in tarnation?