| ▲ | Alpha3031 5 hours ago | |
That's very interesting. Does that mean you can reduce say, a 30B class Q8 from ~30 GB down to 10 GB or less? | ||
| ▲ | withinboredom 2 hours ago | parent [-] | |
704gb -> 564gb; 358 gb -> 270 gb; 28.79 gb -> 7.65 gb; 439 gb -> 93 gb It depends on the total entropy of the model. Smaller models have less entropy. | ||