I’m always curious why local models aren’t being pushed more for certain types of data the person is handling. Data leakage to a 3rd party LLM is top on my list of concerns.

Worth noting that AWS Bedrock makes it easy to have zero retention with premier claude models. Not quite local, but it feels local-adjacent for security while getting affordable access to top-performing models... GCP appears to be a bit harder to set this up.

I am not as concerned with that with API usage as I am with the GUI tools.

Most of the day gig is structured extraction and agents, which the foundation LLMs are much better than any of the small models. (And I would not be able to provision necessary compute for large models given our throughput.)

I do have on the ToDo list though evaluating Textract vs the smaller OCR models (in the book I show using docling, their are others though, like the newer GLM-OCR). Our spend for that on AWS is large enough and they are small enough for me to be able to spin up resources sufficient to meet our demand.

Part of the reason the book goes through examples with AWS/Google (in additiona to OpenAI/Anthropic) is that I suspect many individuals will be stuck with the cloud provider that their org uses out of the box. So I wanted to have as wide of coverage as possible for those folks.

but they claim your data is private and they will totally not share any of it with their advertising partners!