https://frontierharness.org/ allegedly this is on the Pareto frontier, though I don't know how good of a benchmark this really is. it seems to focus on one-shot type tasks, whereas the real utility of one harness over another seems to make itself known in long running tasks.

also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.

▲

zihotki an hour ago | parent | next [-]

I wonder why there is no Github Copilot in the comparison. It also supports BYOK and is quite capable

▲

polotics 3 hours ago | parent | prev | next [-]

how could the authors of frontierharness.org not see they used the exact same symbol for Claude Code and OhMyPi?

	▲	hypfer 3 hours ago \| parent [-]
		The authors might lack vision capabilities? It happens more often than you think, especially when VRAM is limited!

▲

cdelsolar 2 hours ago | parent | prev [-]

i've seen "Pareto frontier" literally 500 thousand times in the last two weeks or so, and maybe once or twice before that. Can someone please explain what happened recently

▲

zinoc an hour ago | parent | next [-]

In case people need a refresher on what the Pareto frontier is, this article does a great job explaining it (with Mario Kart!): https://www.mayerowitz.io/blog/mario-meets-pareto

	▲	pu_pe 27 minutes ago \| parent [-]
		Everyone started using the term after that article came out, it feels like...

▲

233mhz 2 hours ago | parent | prev | next [-]

It sounds fancy and technical. Like when people use "orders of magnitude" six times per thread on this forum but no ever ever use it irl

▲

vrganj 2 hours ago | parent | prev [-]

People realized AI is expensive and wanted a cool techy way to say "good bang for your buck"

	▲	zmmmmm 34 minutes ago \| parent [-]
		"my model sucks but it does so cheaper than anybody else"