The catch is that it's not really doing anything. It "stores" 16GB of quants by... storing the base 2GB model and quantizing it on user's machine on demand.

The extra diff files are only there because the quantization is not fully reproducible*, and author's 3-line llama.cpp fix PR** supposedly fixes this. With it fixed, then in my understanding the author's tool is literally just a wrapper around llama-quantize.

* I also wonder if this even matters for quant quality. In fact, isn't it possible for contracted FMAs to be more numerically accurate (even if less correct from IEEE pov) than non-contracted math?

** I'd give the PR like 50% chances of not surviving the day, since AFAIK on llama.cpp, vibed PR descriptions usually end up rejected as a rule. Maybe the diff being super trivial could save it from this fate, dunno.