▲ | CamperBob2 7 days ago | |||||||
Are you talking about the guy in Temecula running two different auctions with some of the same photos (356878140643 and 357146508609, both showing a missing heat sink?) Interesting, but seems sketchy. How useful is this Tesla-era hardware on current workloads? If you tried to run the full DeepSeek R1 model on it at (say) 4-bit quantization, any idea what kind of TTFT and TPS figures might be expected? | ||||||||
▲ | oceanplexian 7 days ago | parent | next [-] | |||||||
I can’t speak to the Tesla stuff but I run an Epyc 7713 with a single 3090 and creatively splitting the model between GPU/8 channels of DDR4 I can do about 9 tokens per second on a q4 quant. | ||||||||
| ||||||||
▲ | justincormack 6 days ago | parent | prev [-] | |||||||
Tesla doesnt support 4 bit float. |