| ▲ | joe_the_user an hour ago | |
Quized ChatGPT about the article... I had trouble seeing it in the article (it's hidden in footnote 3) but it seems perplexity/log-loss is tracked using the model's ability to track an unreleased code base of OpenAI. The log-loss involved in code has declined more for code than prose in all of the models but has supposedly declined everywhere. [1] At the same time, the decline of the latest models on a common sense benchmark might be meaningful: https://news.ycombinator.com/item?id=50588916 | ||