My paranoia tells me they A/B test users which means consistent benchmarks go through the high performance pipeline while regular requests fall back to the quantized / experimental / low cost version. I can see the main reason being the oldest one in the book, which is making more money.

It could also be a regional effect where systems are under higher load and hence dynamically routing to / provisioning lower precision instances; or some other factor where one is affected whereas someone else isn't.

It's not paranoia and would align with https://www.reddit.com/r/ClaudeCode/comments/1xr9x5y/the_ner...

I wonder as well whether would this pop up on nerfbench