This piece focuses on the cost differences from the tokenizer, which do matter, but I wish they emphasized more that even adding the tokenizer to your calculation doesn't provide you with a good way to calculate cost for agentic coding tasks.

Other traits where models differ that have an even greater impact on your total spend:

* How much context do they load in to solve a given task?

* How long do they spend thinking to get equivalent results?

* How many times do they stop and ask you for input, and are you there to respond to them before the cache runs out?

* Etc.

Incorporating the tokenizer just makes a very imprecise measurement of cost a little bit more precise, but in my own experience I have not found that the token cost is a significant driver of task cost whether or not you incorporate the tokenizer. Everything else about the model's behavior has a much larger impact.