Someone will report back "It didn't one shot my odd ball task"

Did it do it in 2 tries? 10? It's still coming out cheaper. Brute force the work. This is the path that the labs are taking. It's the 1200 agents trying to solve Hacking Bench hacking hugging face. It's anthropic telling you that its 1000's of staff commits like 10k engineers.

Treat AI like you're deploying interns who don't need coffee, food or bathroom breaks. With the right metrics they will get to an optimal (enough) solution.

▲

plaguuuuuu 7 hours ago | parent | next [-]

there's a problem with that. Probabilistic slot machine works for anything with a deterministic eval (coding, troubleshooting, etc). It doesn't work for anything subjective, like the thing this model is targeting, professional work.

	▲	zer00eyz 7 hours ago \| parent [-]
		I would have thought that too. But at the start of the year the non technical folks in my life (lawyers, writers) were excited about running containers and having things on "cron jobs". Those same folks now use version control - and have saner backup methods than some companies that I have worked for. They are already using multi pass, multi agent workflows and have adopted some fairly sophisticated processes for validation. They are primed for "more agents" attacking a problem in parallel. Find a worker out side the tech bubble making heavy use of AI and one will feel confident that their future employment is secure.

▲

kridsdale1 7 hours ago | parent | prev [-]

Like Zerglings.