I don’t think so, the HF weights are bf16 which means 24GB + cache/overhead.

It sounds like marketing spin where the performance claims are based on BF16 and the “runs in 16GB” claim is on a totally different quantized version.

[flagged]