How is 30B smaller than 27B?

LeBit 3 hours ago | parent | next [-]

It uses fractal compression

lostmsu 2 hours ago | parent | prev [-]

They say it is trained with quantization awareness, so it should only be 15GB or so. Qwen was only trained in FP8 with QAT.

UPD, NVM, got misled by comments here. It is actually almost 60 GB so much larger

	▲	ricardobeat an hour ago \| parent \| next [-]
		Quantization awareness doesn’t change the size of the weights, just means it won’t degrade when quantized. QAT = quantization aware training. They will both be very similar in size at the same quant.
	▲	xienze an hour ago \| parent \| prev [-]
		You're mixing up sizes of different quants. The 60GB is unquantized, and Qwen's unquantized size is around 54GB. Their sizes as like quantization levels are similar.