You should know - for coding they make terrible mistakes as well.

But programmers have this concept of a "code review" where another person looks at the code to look for problems.

We use this same technique with our LLMs. Most mistakes are caught by having a second LLM look at it. Doesn't even need to be a different model - just make sure it has a different context.

▲

int_19h an hour ago | parent | next [-]

> Doesn't even need to be a different model - just make sure it has a different context.

I find that the number of issues discovered is noticeably higher if you do use different models though. I'm doing some very finicky things (formal semantics) and find that there's value in review panels as large as 5 different models from different families. It gets even more profitable if you set it up as a truly agentic panel where after writing their own separate reviews they get to see what others have said, and adjust their positions or defend them etc. Some models are not that good by themselves but can be surprisingly good at finding flaws in others' reasoning (Grok for example).

▲

flyinglizard 3 hours ago | parent | prev [-]

Programming has a long standing culture of accepting the code to be somewhat wrong, so we have various tests, linters, reviews and error handling. Also in programming there are many ways to do something and it's the end result that matters most.

Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?

▲

jll29 2 hours ago | parent | next [-]

You could say the Salvatorian Clause in contracts is like exception handling: a "catch (all)": even if some clauses in this contract are illegal, the remaining contract stays in place.

Logically, this actually doesn't make sense strictly speaking because the sentence creates a paradox: doesn't it make clear whether it includes itself or not, and each reading ends up in trouble. There is a "tradition" in law around the world to accept the only benign reading of such clauses, which I always found funny given that in all other ways lawyers adopt the most adversarial mindset imaginable.

▲

qarl 3 hours ago | parent | prev [-]

I don't understand. I just provided advice about getting better output. Are you trying to reply to someone else?

	▲	flyinglizard 3 hours ago \| parent [-]
		You made some points worthy of expansion: >> You should know - for coding they make terrible mistakes as well. >> But programmers have this concept of a "code review" where another person looks at the code to look for problems.