| ▲ | Tiberium 6 hours ago | |||||||
There's also a tokenizer efficiency difference: modern Claude's 100K tokens are about ~60-65K modern GPT tokens, so in reality the Luna cutoff is much further away than the Haiku one. You can test with Anthropic's count_tokens endpoint or with https://crates.io/crates/tokwc | ||||||||
| ▲ | AtNightWeCode 6 hours ago | parent [-] | |||||||
> ...this tokenizer, the same input text produces approximately 30% more tokens on Claude Haiku 5.5 than on Claude Haiku 4.5. So, it is might be even worse. | ||||||||
| ||||||||