| ▲ | kingstnap 3 hours ago | |
Beating even a very small LLM at next token prediction is basically impossible. [2212.11281v2] Language models are better than humans at next-token prediction https://arxiv.org/abs/2212.11281v2 We knew this many years ago. Like GPT 1 era. Actually you can play it here. The game is pretty frustrating ngl. And I can clearly see why induction heads are a thing. Because when a word repeats you guess is to just guess the same thing from before. Language modelling game https://rr-lm-game.herokuapp.com/ | ||
| ▲ | rfgplk 3 hours ago | parent [-] | |
Will try this out, thanks | ||