We have a lossless compression codec (working on open sourcing it over the next couple of weeks) that reduces it down to its minimum entropy -- it cannot be compressed further. On all tested large models, it's a ratio of 1.34-1.23 -- and smaller models up to 3.76x. It also increases the effective bandwidth by the same rate.

▲

xscott 4 hours ago | parent | next [-]

Exploring compression algorithms for weights is a good idea, and I hope you have a successful product. However, if you can prove this statement:

> reduces it down to its minimum entropy -- it cannot be compressed further.

I think you could make a lot more money elsewhere :-)

https://en.wikipedia.org/wiki/Kolmogorov_complexity#Formal_p...

	▲	withinboredom 2 hours ago \| parent [-]
		We're not an AI company ... nor do we have any reason to use it. Just a fun idea that was fruitful.

▲

Alpha3031 5 hours ago | parent | prev | next [-]

That's very interesting. Does that mean you can reduce say, a 30B class Q8 from ~30 GB down to 10 GB or less?

	▲	withinboredom 2 hours ago \| parent [-]
		704gb -> 564gb; 358 gb -> 270 gb; 28.79 gb -> 7.65 gb; 439 gb -> 93 gb It depends on the total entropy of the model. Smaller models have less entropy.

▲

kaxu-64 6 hours ago | parent | prev [-]

[flagged]

	▲	withinboredom 5 hours ago \| parent [-]
		Thanks for your opinion