It really is a very simple setup. I basically had an old Intel based Mac mini from 2020. The intel chip inside it is from 2018). It's a 3 GHz 6-core Core i5. I had upgraded the ram on it to 32 GB when I bought it. However, the ollama only uses about 5.5 gigs of it. So it can be run on 16gb Mac too.

The Qwen model I am using is fairly small but does the job I need it to for classifying headlines pretty decently. All I ask it to do is whether a specific headline is political or not. It only responds to me with in True or False.

I access this model from an app (running locally) using the `http://localhost:11434/api/generate` REST api with `think` set to false.

Note that this qwen model is a `thinking` model. So disabling it is important. Otherwise it takes very long to respond.

Note that I tested this on my newer M4 Mac mini too and there, the performance is a LOT faster.

Also, on my new M4 Mac, I originally tried using the Apple's built in Foundation Models for this task and while it was decent, many times, it was hitting Apple's guardrails and refusing to respond because it claimed the headline was too sensitive. So I switched to the Qwen model which didn't have this problem.

Note that while this does the job I need it to, as another comment said, it won't be much help for things like coding.