Remix.run Logo
amitport 8 hours ago

r is a single value per vector. You don't have to quantize it, you can keep it and quantize the billion+ other coordinates of the vector.

mungoman2 8 hours ago | parent [-]

What they're saying is that the error for a vector increases with r, which is true.

Trivially, with r=0, the error is 0, regardless of how heavily the direction is quantized. Larger r means larger absolute error in the reconstructed vector.

amitport 7 hours ago | parent [-]

Yes, the important part is that the normalized error does not increase with the dimension of the vector (which does happen when using biased quantizers)

It is expected that bigger vectors have proportionally bigger error, nothing can be done by the quantizer about that.