One thing I have been using LFM 2.6 for is annotating emails, text messages, and other metadata. I keep it all ingesting into one giant database and annotate it all as it comes in on an RTX A4500. The LFM annotations are basically triage, and then slowly annotations trickle into my halo model based on lowest confidence (right now that's DeepSeek 4.1 Flash 1017) for an upgraded annotation. (Agents can then search this database for context).

Excited to roll out LFM 3 to replace it, looks like a big step up. Right now LFM 2.6 gets trustworthy annotations on just over 40% of data points. Just waiting for these to get into LocalAI.

You might also consider the decision models, d1-2B is very strong and good for calibrated classification. Would be happy to share links/resources if you're interested.

You're doing something similar to what I've been prototyping for the automatic metadata enrichment part. So far I've been using deterministic older techniques + manual for my personal data stream, but I'm trying to learn what the state of the art is for efficiently annotating data with LLMs and newer ML.

This seems interesting, but if anyone else is also doing a similar thing, I'd appreciate tips on how to approach it. So far, I think I have some small RAG model in a demo or the very small qwen models. I think fine tuning seems interesting too. Some of the decision models applied in layers (like a binary search on a pre-generated corpus) could be fast enough to help too?

idk too many options, i'm sure this is a solved problem lol

That's very cool. I'm curious if you need a GPU for your use case (email and text triaging). There are some open source alternatives[0] that run/train on CPU, so you can self-host or run locally (and train on your own data for better accuracy), would something like that work for you?

0: http://jeffyclassify.com/

I can highly recommend gliner2.5, been using it for tagging news articles, fine tuned on my home 5090 in a few minutes on 500 articles tagged by gpt 6.1, works fantastically. Deployed on hetzner running on a cpu .

Have you considered fine-tuning on your dataset?