| ▲ | Havoc 2 days ago | |
Is there something different about that accuracy measure? i.e. relative to perplexity term usually used Going from BF16 to 2.8 and losing only ~5% sounds odd to me. | ||
| ▲ | kouteiheika 2 days ago | parent [-] | |
It's accuracy across GSM8K, MMLU, IFEVAL and LiveCodeBench. They detail their methodology here: https://byteshape.com/blogs/Qwen3-4B-I-2507/ | ||