So excited to see this - the big advantage of 1.58 bits is there are no multiplications at inference time, so you can run them on radically simpler and cheaper hardware.

▲

Animats 9 hours ago | parent [-]

At 4 bits, you could just have a hard-wired table lookup. Two 4 bit values in, 256 entry table. You can have saturating arithmetic and a post-processing function for free. Somebody must be building hardware like that.

	▲	londons_explore 3 hours ago \| parent \| next [-]
		Low level software engineers use look up tables. Hardware engineers realise that a compiler will almost always find some combination of gates which is smaller/faster than the contents of any table.
	▲	AlotOfReading 7 hours ago \| parent \| prev \| next [-]
		A LUT is pretty wasteful. You only have a one bit significand, so the mantissa and sign bits are boolean binops, and the exponent is a 2 bit adder.
	▲	Taniwha 7 hours ago \| parent \| prev [-]
		and so you can at 1-bit too, and the hardware will be even smaller and cheaper