Any ideas what decider models are actually good? There does not seem to be a strong competitive benchmark culture (as with LLMs) for whatever reason?

walls 4 hours ago | parent | next [-]

That's good but I would definitely appreciate one with cost to run the benchmark and cost vs accuracy breakdowns.

Artificial Analysis has an Index which also includes Cost per Task, Time per Task, accuracy, calibration etc. here: https://artificialanalysis.ai/evaluations/artificial-analysi...

(caveat, I work at AA and helped build this Index)

	▲	talloaktrees 2 hours ago \| parent [-]
		just found this today and used it as one of the links I sent a coworker who hadn't heard of decision models yet!

[dead]