People will eventually realize that this is exactly what AI does for most workflows. The propaganda is "synthetic humans", "obsolete mathematicians" and so forth, but what we will likely see is a fantastic time saver for things we didn't like to do to begin with.

I was thinking about this today. I am definitely making things easier but.. if I was paying the actual costs of this service, could it even be close to worth it?

Well now that Astra for Law is here I give it about five minutes before social media grifters claim to be 10x lawyers

Agreed. However:

That's a factor of 3 to 5 improvement reported on here. If you keep stacking these kinds of improvements, you end up at month to minutes pretty quickly.

This is really where Gemini still excels. Honestly from my experience the others aren't as good.

Yep this is one of the best use-cases for a fine-tuned SLM/SVLM.

I'm wondering if the bottom is going to fall out of frontier models when people start to realize this. Sure, as an experienced programmer you can tell the difference between a compact free model and the ones that are 100x bigger and cost billions to train, but 90% of busy work is dead simple: turn a PDF into json fields, or do the inverse and fill out a form. As the tools for this conversion become more widespread you can also imagine an economic shift toward efficient information brokers that make frontier models less necessary.

It's also not really clear if everyone is going to want a frontier model when the real implications sink in. Maybe we'll get sick of incomprehensible code optimizations and people wile tire of reading AI prose that feels ever-more-human. There might be a few use cases, but who is going to pay for this when providers start charging enough to be profitable.

Are they? I've had excellent success. The confusing part of this is that there are two types of PDF. The first is a "normal" digital PDF. The second is a scanned PDF. The first can essentially be read like a document. LLMs have no issues with this. It's the second kind of PDF where the constraint becomes the vision capability, and this is very impressive with Astra. I've had no issues with either. I imagine there could be issues with unusually dense and/or misaligned text on scanned PDFs, but I have not tested this.

The bottom line, though, is that PDF OCR is usually regarded as a solved problem. LLMs won't usually do the recognition itself. It will farm it out to established tools which are very good.

Yes but this is more an artifact of how much PDF sucks as a file format than LLMs being crap.

Which models are you using.

they were in 2023

Yeah, it's sort of odd that even here people are pushing back like it couldn't possibly be doing this, you must have misunderstood it or are overhyping it.

I come neither to praise AI nor to bury it. This is a case where the tool is useful.

The key is that this is work the attornies themselves do not want to do and which does not require their brains or expertise. It's not just "boring" but a bad use of their time.

"Sounds kinda boring" - many do say this about lawyers.

Yes, remember that these are effectively random PDFs in various different designs and formats, some of them not editable or even OCR'd.

It took a human attorney 20-30 minutes on average to manually copy-paste data from these PDFs into a spreadsheet (while also fixing any errors they found in the document and re-checking for quality).

Now, the AI copies everything into the spreadsheet in a small amount of time, and then the human reviews it. It takes maybe ~5-7 minutes to scroll to the appropriate pages in the document, read the lines vs the spreadsheet, and make corrections. So you've gone from 2-3 items an hour to ~8-10 items an hour.

Maybe you could pay someone to develop an OCR/ML application that could do this. But that project would never be profitable, even with the time savings. At the cost of a couple Claude subscriptions, it makes sense.

You don’t need a trained attorney to schematize data. The LLMs are used to make the data easier to understand and manipulate.

Reviewing something takes less time than producing it.

I don't think the details of the company's benefits plan is considered your private information. But I am concerned about errors in this parsing leading to incorrect results (and that compounding everywhere throughout society when this sort of workflow becomes ubiquitous).

These documents are not personal private information. They might be client privileged in some cases? But I don't think so.

>Non legal background people trying to avoid going to a lawyer. They should definitely NOT use an LLMs.

I used LLMs to replace a lawyer this year to great effect. I successfully advised myself as to the right strategy and drafted an immigration petition in the correct language. Granted it wasn't 100% LLMs (I also supplemented it with "traditional" online research and wrote the letter myself using the LLM version as rough guidance), but still, I certainly would've gone to a real lawyer in 2024 for this stuff but didn't.

Professional advisory work can be divided into two types:

1. Once you know the rules of the game, the next move is immediately obvious.

2. Knowing the rules is not enough to know the next move -- that requires judgement and experience.

LLMs in the hands of someone smart and savvy can easily replace the first type of work.

> Non legal background people trying to avoid going to a lawyer. They should definitely NOT use an LLMs.

In my experience, it is incredibly hard, practically impossible, to find a lawyer who will even talk to you. I think this is a valid use case to get at least some understanding what the next steps might be and if it even makes sense to seek legal representation.

Might be true today, but I still feel pareto principle applies. These points also sound like gatekeeping a bit and I hope future AI versions prove that law is also essentially something that can be made as programmable as possible for most common answers if the underlying principles and constitution are organised such.

If there is no precedence for X happening when Y , Z, P, Q, R are special combinations , the judge decides and sets precedence. But if another judge interprets it differently with similar set of circumstances it is not considered incorrect I presume especially if in different jurisdiction unless overridden by a higher court. This is my layman understanding but I think this design is wrong and essentially where human subjectivity controls the system.

Subjectivity is not totally avoidable in law but the goal of the legal system should be reduce it as much as possible along with ambiguity.

I do think non legal background folks can use LLMs today to sense check legal ideas, like for instance, "what are my legal rights in this situation?"

These situations usually are not ones that an individual can justify the time or money to contact an actual lawyer, but then if they do decide to contact one they will come in with better questions and more of a sense of what they are expecting.

This is similar to medical. Should you use LLM to diagnose yourself, treat yourself with prescription drugs you buy from shady gray market online sellers? No. But you can use it very well to know when it's time to go to the doctor and what to ask.

You sound much like software engineers sounded when the first coding agents dropped.

"You can't trust the output, it doesn't understand bigger systems"

"Its an art, you need to learn the ropes of it to truly write good code"

Its a very dangerous line of thinking. Software engineering will never be the same, as writing code has basically vanished from the daily workflow. Not for every specialized usecase, but for many.

You can replace "laywer" with "software engineer" in your post and it holds equally true. I think this also applies to other factions.

> There is a reason why interns are made to do grunt work. The "chain of command" is built so they learn on basic stuff, learn to make decisions, etc. Without these basic decisions under their belt, an intern can't take bigger decisions later on. They just won't know....

That might be true, but that doesn't mean you benefit from training up juniors.

>I see 3 kinds of potential users.

3 kinds of users that don't scare the shit out of you.

My buddy works with lots of folks that have gone all in on Harvey. There are other kids of users.

There are 3x as many lawyers per capita in the US as we had in the 1970s. Credit agreements that used to be 50 pages are now 500. This increase in volume has not lead to any increase in actual value. The legal profession is a metastatic cancer on our society. Un-fuck this situation and maybe I'll think about taking your advice of not using a LLM.

[dead]

I'm thinking back to some of my interactions with corporate lawyers and you know what? My bar (<- pun) for trusting Astra more than a lawyer is pretty low.

The proof would be that the attorney who did the work before, and who still reviews all ingested data, says it is.

You should free your dog then because a creature that intelligent should not be anyone's pet.