Remix.run Logo
cpburns2009 4 hours ago

Only VRAM? Unlikely unless you can also load the whole model into regular RAM. The previous 3.5 release was 250gb at BF16, so FP8 would likely be around 125gb. Your best best is FP4/Q4.

isatty 2 hours ago | parent [-]

Very sad. I try not to go below q8.