Once this weight format war settles down, hardware can be built to support it. Presumably you want matrix multiply hardware optimized for whatever weight format turns out to be reasonably optimal.

Optimization is post hoc here : you have to train first to be able to huffman en ode, so it's not a pure format question