Remix.run Logo
monocasa 2 hours ago

These are trits, which provide their own efficiencies.

Interestingly, a trit x float multiplier is cheaper than a trit x integer multiplier in hardware if you're willing to ignore things like NaNs.

0 and 1 are trivial, just a mux for identity and zero. But because floats are sign-magnitude, multiply by -1 is just an inverter for the sign bit, where as for integers you need a bitwise inverter and full incrermenter.

buo 35 minutes ago | parent [-]

Do you know a good reference to learn more about this (quantizing weigths to 1.58 bits, and trit arithmetic)?