| ▲ | moron4hire 8 hours ago | |||||||
You're counting just the egg-having receipts, but there were over 11 thousand receipts they had to go through to get to that 500-ish subset. I'm assuming OP wanted to process all of the receipts and then selected just eggs for a simple analytics job. With your rates, the human would cost almost $2000. | ||||||||
| ▲ | cheschire 6 hours ago | parent [-] | |||||||
Capturing the egg price from known egg receipts was the problem I was focused on, but you're right that there was also a filtering problem in the original spec. You get my upvote for continuing to make the problem interesting for me! Had the filtering been done during the initial document storage, then the cost would have been much cheaper than your $2,000 estimate. Essentially binning the receipts based on "eggs" or "no eggs" would be free. But, crucially, what happens when the question changes from price per egg to price per gallon of milk? Now the whole stack would need to be sorted again. The $2,000 manual classification would need to be re-applied. Isn't traditional ML-based classification cheaper for this problem at industrial scale than an LLM though? The OP did of course attempt more traditional generic off-the-shelf OCR tools, but let's consider proper bespoke industrial ML. Just as a off-the-cuff example, I would probably start with building a tool that locates the date/time from a receipt and takes an image snip of it. Running ONLY image snips through traditional OCR is more successful than trying to extract text from an entire receipt. I would then train a separate tool that extracts images of line items from a receipt that includes item name and price. Yet another tool could then be trained to classify items based on the names of the items purchased, and a final tool to get the price. Now you have price, item, and date to put into your database. Perhaps generating the training data to train the item classifier is the only place I could see an LLM being more cost effective than a human, but classifying tiny image snips is not the same as one-shotting an entire receipt. As an aside, if there's any desire to discuss how expensive training ML is, don't forget the price to train an LLM as well. All of this is to say I believe traditional ML is the solution. I'm still not seeing the value prop of LLMs at the industrialization scale outside of very targeted training data generation. A more flippant conclusion might be that we can replace a lot of the parts of data science that makes PhD types get bored with creating traditional ML solutions. | ||||||||
| ||||||||