| ▲ | CamperBob2 4 hours ago |
| The main barrier to wider spread adoption of this right now is that very few people have the legit on-prem hardware resources/budget to run something like an abliterated Q8 of deepseek v4.1 flash 1017 Not as bad as you might think. This costs about as much as a decent new car nowadays. The abliterated 1017 release I've seen is crazy good. |
|
| ▲ | walrus01 4 hours ago | parent | next [-] |
| I guess it depends if we're talking about decent new car at $40,000 or $95,000? I've seen lots of people claiming they're running v4.1 flash 1017 on local hardware but then it turns out it's something like the unsloth Q2 XXS or similar size. The Q8 is 508GB so assuming for a moment we're not running it on a Mac, we're looking at a cost of five or six 96GB GPUs to load the whole thing with a good sized full context and KV cache. |
| |
| ▲ | hgoel 2 hours ago | parent | next [-] | | Haven't bothered with DS4.1 personally but I've seen reports of running it with native weights on 4x DGX Sparks. Currently ~$30k but last month that setup was ~$20k. | |
| ▲ | CamperBob2 3 hours ago | parent | prev [-] | | 4x 96 GB runs the NVFP4 quants, which are indistinguishable from Q8 in real-world performance for this particular model. That's not always true but DeepSeek is unusually quantization-friendly. Also remember that a big chunk of the model consists of engrams that can live in system RAM or even on an SSD if necessary. I imagine the various GGUFs are still pretty impressive, but haven't tried them myself. |
|
|
| ▲ | KellyCriterion 4 hours ago | parent | prev [-] |
| > new car ...which is still tooo much for most people to spend on "AI hardware".... |
| |
| ▲ | CamperBob2 3 hours ago | parent [-] | | More like, the ability to chain a demon in your basement that does what you tell it and can't be rug-pulled by his bosses in San Francisco. | | |
| ▲ | hgoel an hour ago | parent | next [-] | | AI is the endgame for the joke that computing is the act of etching arcane signs into crystal and imbuing them with lightning. | |
| ▲ | walrus01 3 hours ago | parent | prev [-] | | Recently saw a hand drawn cartoon to the effect of, a person is asking Claude or chatgpt to do something, it refuses, so the next step is "make the Chinese one do it", and the Chinese model is represented as a chained demon | | |
|
|
