Remix.run Logo
mrugge 9 hours ago

1. Efficient recursive transform of kv embeddings into polar coordinates 2. Quantize resulting angles without the need for explicit normalization. This saves memory via key insight: angles follow a distribution and have analytical form.

quotemstr 8 hours ago | parent [-]

Reminds me vaguely of Burrows-Wheeler transformations in bzip2.