| ▲ | rayiner 15 hours ago | |
I had Claude evaluate my vibe-coded extractor against yours on the EcoFactor corpus, which I didn't test on, and an internal corpus I have: > The EcoFactor result inverts. There, on hand-authored gold over 21 pages of clean native text, pdfgeo won the accuracy checks 25/42 to 19/42 — its geometry work on multi-column patents, table cells and timestamp lanes is genuinely better, and nothing here contradicts that. But this corpus is 21,652 pages of mostly scanned, OCR'd, skewed exhibits, and at that scale pdfgeo's robustness is the binding constraint: it loses 6.4% of the corpus to two trivial crashes and an over-strict rotation test, duplicates text on 34 files, and drops 80% of wrap hyphens. lawpdf extracted every page, 7× faster, with 2.5× less furniture leakage. It looks like the actual geometric analysis came out the same general shape, which is an ad hoc heuristic with hard-coded values attempting to figure out where the gutter is located, etc. Seems like both have some overfitting to the text data set. | ||
| ▲ | kolinko 3 hours ago | parent [-] | |
If you have more test data I can try this with my harness - you can email me at kolinko@gmail.com btw :) Thanks for the benchmark, I was seriously looking forward to it! Would you consider such even results good for a one-shot? I wonder how well my harness performs :) | ||