Remix.run Logo
▲ pishpash 2 hours ago

There are more general information measures, Rényi entropy, von Neumann entropy, etc. for different system setups, but for token-by-token information learning and prediction within classical ergodic processes Shannon entropy is where it's at.