> the 122B version of 3.5

Yeah, this is what I'm holding out for, the NVFP4 variant of 3.5 122B is blazing fast with reasonable quality and even with max context fits perfectly within 96GB.

SO MUCH THIS. I have Strix Halo with 128GB RAM and was a large and fast model like 122B A10B. Here's hoping!

Absolutely. There's a glaring gap in the space for something about the size of Nemotron Super or just under, but actually ... competent.

The fantasy is a 100B or 80B model, but MoE and highly tuned for coding.

Indeed! That would be the sweet spot for my 2x3090 rig

> I'm afraid that small capable models have been left in the dust

I wholly disagree. Rather than going the "everything is a claude code skill" route, I've been hacking together purpose-built harnesses for all sorts of tasks, and in that environment a wee little baby model can do some really useful things. You end up burning lots of tokens making the thing, but then all that investment comes back when the resulting tool works perfectly fine on a dinky little model that fits on my 3060 Ti.

Google makes Gemma 4 31B QAT; that's not a small lab. It's one of the better models out there for consumer hardware. Allows me to run it on a 7900XTX with 64k context.

nvidia and amd don't give a flying fuck about end users right now while they can milk triple digit markups from infinite money VCs via data center GPUs.

someone will keep putting out consumer level models. Once you have the larger models, you can derive the smaller onces.

Europe will definitely be interested in democratizing these things if China starts losing interests; from there, there'll be more countries looking to keep their citizens entrained in their own Country's infrastructure.

It'll especially be true if the memory cartel keeps prices high and NVIDIA tries to gouge higher memory models.

It's an arms race everyone can join because PC hardware was mostly democratized in the last decade.

there's been a lot of research about reducing models by taking out layers; there's also using it to train smaller models by optimizing parameters.

I dont see most model building as anything more than a pig at a slop troth, despite the level of sophistication; they're still rarely pruning the input beyond random sampling.

I run a 2x 3090 rig, but a single 3090 already provides a great experience at a reasonable quant and context size. On a single card I used to run Qwen_Qwen3.6-27B-Q4_K_M or similarly quantized 35B MoE at 65536 context size