Whoah, could this mean we can treat layers like a jpg, where we come up with a formula that estimates the weight values of a layer instead of storing all of the weights?

This is more or less what those lovely 1.5 bit quantisations are already doing