you use the wrong word

the Anthropic tokenizer is not worse, its more expensive/verbose

So, worse? Because we benchmark off token use when talking about token use, and everyone else understood that.

	▲	marcosdumay 3 hours ago \| parent \| next [-]
		The most important feature of a tokenizer is dividing the inputs into independent values that the neural network can work with. It's not the size.
	▲	jascha_eng 4 hours ago \| parent \| prev \| next [-]
		I mean it might lead to better performance on the model side. So the tokenizer is better but more expensive.
	▲	otikik 6 hours ago \| parent \| prev [-]
		It’s better for them