Remix.run Logo
▲ vova_hn2 3 hours ago

> token usage

It is not exactly accurate, because it ignores speculative decoding which can often save a lot of time, compute and energy.

See this branch: https://news.ycombinator.com/item?id=50441444#50497854

▲rhdunn 2 hours ago | parent [-]

Speculative decoding will be model/software based and apply to all languages. The time, compute, and energy will be less compared to other models, but for the same model, hardware, and software used my reasoning still applies -- comparing English to other languages (unless optimized by the tokenizer) then English will consume less tokens and thus less energy. That is, it is still a function of the tokens used.