| ▲ | GIFtheory 2 hours ago | |
The post makes perplexity seem overly mysterious… First, “log perplexity” is just cross entropy. Why not call it that? Also, when perplexity is this low, it arguably makes more sense to report perplexity. A perplexity of 2 basically means that the model can guess a set of size 2 that contains the true next token. | ||