That's kind of exactly what floats are. You store the log2 of the number, a bit for its sign, and in what remaining bits you have left some fixed-point scaling between adjacent powers.

No, that is not what floats are. A logarithmic number system literally just stores the logarithm of a number (and a sign bit) and manipulates it as a logarithm. The significand is 1, always. So multiplication & division are simply addition & subtraction, respectively. But this simplification for multiplication, division, roots, and powers is counterbalanced by more complex addition and subtraction.[1]

[1] https://en.wikipedia.org/wiki/Logarithmic_number_system

I think they mean storing just an exponent instead of an exponent + mantissa. Makes multiplication much easier to implement.