Remix.run Logo
Bnjoroge 7 hours ago

Using tokens to evaluate models is an outdated approach. Cost per task is what matters. Not all tokens are created equal

WhitneyLand 3 hours ago | parent | next [-]

It’s not outdated at all to use tokens to estimate performance, it’s directly related.

drob518 6 hours ago | parent | prev [-]

Yes, but… more thinking tokens also means longer solution generation time. That said, v4 Flash is a fast model. I use it all the time because it’s very smart for the price. But it is verbose sometimes.

kzrdude 6 hours ago | parent [-]

The thinking trace was (preview) frustrating to read, I think I'd prefer a summary view of it at this point.