LLMs are the first genuinely useful legal tech since the Internet. I'm pretty shocked, though, at the delta between how competent Claude is on code versus legal work. It's good for research and data organization, but terrible for drafting. I wonder if this is a structural problem with the lack of feedback loops. In law, there's no compiler to check for logical or continuity errors in your brief, and there's no unit tests to check for correctness or performance.

Even without that, I think it'll be extremely valuable to clients to allow them to answer simple questions without a lawyer, figure out the lay of the land so they can supervise their counsel, etc.

▲

qarl 2 hours ago | parent | next [-]

You should know - for coding they make terrible mistakes as well.

But programmers have this concept of a "code review" where another person looks at the code to look for problems.

We use this same technique with our LLMs. Most mistakes are caught by having a second LLM look at it. Doesn't even need to be a different model - just make sure it has a different context.

▲

flyinglizard 2 hours ago | parent [-]

Programming has a long standing culture of accepting the code to be somewhat wrong, so we have various tests, linters, reviews and error handling. Also in programming there are many ways to do something and it's the end result that matters most.

Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?

▲

jll29 an hour ago | parent | next [-]

You could say the Salvatorian Clause in contracts is like exception handling: a "catch (all)": even if some clauses in this contract are illegal, the remaining contract stays in place.

Logically, this actually doesn't make sense strictly speaking because the sentence creates a paradox: doesn't it make clear whether it includes itself or not, and each reading ends up in trouble. There is a "tradition" in law around the world to accept the only benign reading of such clauses, which I always found funny given that in all other ways lawyers adopt the most adversarial mindset imaginable.

▲

qarl 2 hours ago | parent | prev [-]

I don't understand. I just provided advice about getting better output. Are you trying to reply to someone else?

	▲	flyinglizard 2 hours ago \| parent [-]
		You made some points worthy of expansion: >> You should know - for coding they make terrible mistakes as well. >> But programmers have this concept of a "code review" where another person looks at the code to look for problems.

▲

samtp 2 hours ago | parent | prev | next [-]

> LLMs are the first genuinely useful legal tech since the Internet

That is an incredible statement that could not be further from the truth. Large scale adoption of email, searchable document databases like Westlaw, LexisNexis, PACER, etc.. , OCR Software, electronic signatures, and tons more have had a much more defineably positive impact on the legal profession since the internet came about.

▲

rayiner 2 hours ago | parent [-]

I think "email" and "westlaw" fairly count as "the Internet." LLMs might be bigger than either of those.

	▲	samtp 2 hours ago \| parent [-]
		The internet came well before both of those. And none of the example I gave have lead to lawyers regularly being fined in court because of "hallucinations"

▲

throwaway20222 3 hours ago | parent | prev [-]

I have found that it’s useful generally speaking to get the intent of contracts and red lines, but actual drafting I agree is where I lose all confidence. My guess is that the significance of the difference between using a word like “and“ or “or“ can be so meaningful that that level of nuance can often be lost. But I know nothing I’m not in the space, I just pay too much money for lawyers.