Remix.run Logo
terhechte 8 hours ago

You can run DeepSeek Flash just fine on 128GB using lower quants. Antirez' DwarfStar supports it really well. The bigger the model, it seems, the less it is affected by high quantization. I know some people are working on quantizing GLM 5.2 so it can fit into 128GB (using additional tricks to select only some experts, etc). There's a lot going on.