Yeah this is an extremely poorly written article. They didn’t even bother to add a “rewrite this article to make it sound less than AI”.

It’s also using a bazillion words to make a point that could be summed up in a single paragraph: there’s a huge variance in the number of tokens required to encode the same content, with code leading the charts.

To be fair, most of this was already known, and Anthropic communicated very clearly about the different tokenizer they started using.

Their compute is also mostly 1:1 correlated to the number of tokens, so I don’t believe in the conspiracy that this is just to inflate prices.