Remix.run Logo
amelius 7 hours ago

Sounds like a side channel attack waiting to happen.

unglaublich 6 hours ago | parent [-]

So I guess we'll all be applying a random rotation to our matrices now to obscure their contents, like TurboQuant does. https://arkaung.github.io/interactive-turboquant/#rotation

buildbot 6 hours ago | parent [-]

Not that it super matters, but random hadamards for quantization have been a thing since way before turboquant.

https://arxiv.org/abs/2404.00456

DiabloD3 3 hours ago | parent [-]

Which llama.cpp now does.