| ▲ | CamperBob2 3 hours ago | |
4x 96 GB runs the NVFP4 quants, which are indistinguishable from Q8 in real-world performance for this particular model. That's not always true but DeepSeek is unusually quantization-friendly. Also remember that a big chunk of the model consists of engrams that can live in system RAM or even on an SSD if necessary. I imagine the various GGUFs are still pretty impressive, but haven't tried them myself. | ||