I think the main problem is surfacing two topics:

- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)

- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?

If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?

▲

tedd4u 3 hours ago | parent | next [-]

OpenAI itself cannot answer these questions. A prospective price-sensitive (or speed-sensitive) user has to experiment to find the cheapest (or fastest) model that is sufficient for each task.

	▲	someguyiguess 2 hours ago \| parent [-]
		Right. Because they don’t know. They’re just pumping out crap desperately trying to stop hemorrhaging billions.

▲

andai 4 hours ago | parent | prev | next [-]

>effort / performance curves by model

Oh, easy:

https://deepswe.datacurve.ai/

Artificial Analysis has their own curves which are pretty similar.

	▲	irthomasthomas 3 hours ago \| parent \| next [-]
		That's good, but they haven't updated it for a month. It is missing all of the current frontier models.
	▲	aftbit 3 hours ago \| parent \| prev [-]
		Seems to be missing all of the 6.0 and 6.1 models?

▲

dist-epoch 3 hours ago | parent | prev [-]

> Is a high effort terra above a low effort Sol

You can literally see this in the charts from the article