This is the thesis behind the "Information Theory, Inference, and Learning Algorithms" course that was taught at Cambridge University.

> Why unify information theory and machine learning? Because they are two sides of the same coin. In the 1960s, a single field, cybernetics, was populated by information theorists, computer scientists, and neuroscientists, all studying common problems. Information theory and machine learning still belong together. Brains are the ultimate compression and communication systems. And the state-of-the-art algorithms for both data compression and error-correcting codes use the same tools as machine learning.

Book (creative commons): https://www.inference.org.uk/mackay/itila/book.html

Lectures: https://m.youtube.com/playlist?list=PLruBu5BI5n4aFpG32iMbdWo...

I had a long ranting comment I deleted. I just don't like this trend of people presenting work in a way that makes you think some combo of 1) they discovered from scratch themselves 2) it's new 3) they didn't try to cite or acknowledge where they learned it/point to good sources 4) they don't really care about trying to teach something deeply, they want shiny stuff that makes them seem deep.

This post references specific parts/calculations, but you'd never know it was not news if you didn't know better.

This is basically a thesis supported by Shannon’s information theory.

Any rigorous CS program should cover this in depth.