As the hype is dying down it's becoming a little bit clearer that AI isn't like blockchain and might be actually useful (for non generative purposes at least)

I'm curious what counts as a midsize model; 4B, 8B, or something larger/smaller?

What models would you recommend? I have 12GB of vram so anything larger than 8B might be really slow, but i am not sure

My take:

Large: Requires >128GB VRAM

Medium: 32-128GB VRAM

Small: 16GB VRAM

Micro: Runs on a microcontroller or GPUs with just 4GB of VRAM

There's really nothing worthwhile for general use cases that runs in under 16GB (from my testing) except a grammar-checking model that I can't remember the name of at the moment.

gpt-oss:20b runs on 16GB of VRAM and it's actually quite good (for coding, at least)! Especially with Python.

Prediction: The day that your average gaming PC comes with 128GB of VRAM is the day developers will stop bothering with cloud-based AI services. gpt-oss:120b is nearly as good as gpt5 and we're still at the beginning of the AI revolution.

It can depend on your use case. Are you editing a large code base and will thus make lots of completion requests with large contexts?