| ▲ | Aurornis 2 hours ago | ||||||||||||||||
> 8 X RTX6000. It will run you around 80-100k to get started 8 x RTX6000 GPUs cost $100,000 alone. You then need to build a system that can support those GPUs with enough PCIe lanes through a PCIe switch. It's going to be $120K to $150K to build or buy a system to run this. | |||||||||||||||||
| ▲ | knollimar 42 minutes ago | parent | next [-] | ||||||||||||||||
isn't throwing that into a [insert financial vehicle that gives 99.99999% safe returns] going to destroy that when you factor in electricity costs? Or even just electricity costs vs token cost | |||||||||||||||||
| ▲ | CamperBob2 2 hours ago | parent | prev [-] | ||||||||||||||||
You can run the NV4FP quant with 8x RTX6000 cards at 50-75 tps output, but not (practically speaking) the OEM FP8 version. You will learn more about PCIe than you ever wanted to know. The real gangstas are running 16x RTX6000s. Too rich for my blood, and the NV4FP quant doesn't seem to be that much worse. | |||||||||||||||||
| |||||||||||||||||