I hope the industry starts competing more on highest scores with lowest tokens like this. It's a win for everybody. It means the model is more intelligent, is more efficient to inference, and costs less for the end user.

So much bench-maxxing is just giving the model a ton of tokens so it can inefficiently explore the solution space.

▲

an0malous 7 hours ago | parent [-]

The premise of the trillion dollars in AI investments is not that it’ll be as good as it currently is but cheaper. It’s AGI or bust at this point.

▲

dcre 16 minutes ago | parent | next [-]

Why is AGI required to make the investments work out?

▲

sosodev 7 hours ago | parent | prev [-]

Yeah, but don’t you agree that less tokens to accomplish the same goal is a sign of increasing intelligence?

	▲	camdenreslink 6 hours ago \| parent \| next [-]
		It could be. Or just smarter caching (which wouldn't necessarily have to do with model intelligence). Or just overfitting on the 95% most common prompts (which could save tokens but make the models less intelligent/flexible).
	▲	energy123 6 hours ago \| parent \| prev \| next [-]
		Less cost to accomplish the same goal is a sign of intelligence. That's not necessarily achieved with less tokens but it may be.
	▲	mchusma 7 hours ago \| parent \| prev [-]
		Kind of? But I really care about price speed and quality. If it used 10x tokens at 1/10th the tokens and same latency I would be neutral on it. Kimmi 2.6 for example seems to throw more tokens to improve performance (for better or worse)