Remix.run Logo
▲ londons_explore 2 hours ago

Is this perplexity per-token or per character or per byte?

Also, is that graph assuming no changes in tokenizer if the measure is per-token?