| ▲ | tjwebbnorfolk 4 hours ago |
| > ~60–76 s/token I don't know if I'd call this "running" |
|
| ▲ | sermah 4 hours ago | parent | next [-] |
| Had the same feeling when I first saw min/km units in some (human) running context. UPD: I know it's not the same at all, just the reversal of units that gets me |
|
| ▲ | tito 4 hours ago | parent | prev [-] |
| I commented similarly below, but as a terrible programmer, I probably perform about 1 minute per token too (at Kimi 3 level). It puts into context how I think about intelligence |
| |
| ▲ | brokencode 4 hours ago | parent [-] | | Maybe in terms of code produced, but one token is only a fragment of a thought for an LLM. It’d be like thinking as slowly as Ents talk to each other in Lord of the Rings. | | |
| ▲ | tito 3 hours ago | parent [-] | | Oh, is that how it works? So, when somebody says a model is running at X tokens per second, it means that the thinking process is running at that, and output tokens are much lower then? Thanks to the explanation. | | |
| ▲ | brokencode 2 hours ago | parent [-] | | It’s all just tokens to the model. Whether it’s thinking tokens or output tokens, they take the same amount of computation to produce. The only difference is whether the token is displayed to the user. |
|
|
|
