| ▲ | zawaideh an hour ago | |
It’s still a bit with only two possible values. But they add a scaling factor to a group of them (128 for example) which when you factor in, results in a fractional number of bits per parameter. | ||
| ▲ | throwawayffffas 8 minutes ago | parent [-] | |
I believe the scaling comes in later, to turn the 1 and -1 into large numbers that may or may not activate the next layer. The way they do it is packing like the other comment says. Each byte represents 5 trinary values instead of 8 binary, and there is a little bit of waste. | ||