Remix.run Logo
Otterly99 15 hours ago

It depends on what you called solved.

If the goal is to only extract the unstructured text from the document, it is definitely solved. Extracting a more natural structure like paragraph separation, tables, header, footers (what is referred as document intelligence) is much more complicated and not fully solved, but I would say almost.

jorvi 12 hours ago | parent [-]

Yup, this.

It is actually one of my test cases for LLMs: take the weekly discount PDFs of all the big supermarkets and process each of them, creating a nice table per supermarkt, converting discounts like 1+1 and only listing discounts that are interesting value. I then share that with a bunch of people.

All models fail this, even the really expensive ones. Even with harness, examples and proper insistent instruction, they'll mix up items and their related discount, which category the item should be in, which page they are on, skipping over items etc.

As said above, you can OCR it, but at that point you're not processing a PDF, you're processing an image.

And yes, I know the underlying raw PDF data is messy, but that's why it's such a good test.