So there's a path to 4 in order to run trillion parameter models? But you can't buy them or fit them in the case? Maybe in the Halo 2...

That all said, running frontier models locally offline would be very nice

▲

ilaksh a day ago | parent [-]

It's over $100k.

▲

Neywiny a day ago | parent [-]

Cheaper than an entry level human. But I've never used a model that size for development so idk if they could replace it. The work I do with even nemotron 550b, it's so dumb it's infuriating. Same with the free Google one (for anything sanitized/generic)

▲

ilaksh a day ago | parent [-]

Try Qwen 3.8 27b or the antirez ds4 0731 quants

Can run on MacBook m5 or new m5 studio or m5 studio ultra or rtx 6000 pro.

	▲	Neywiny a day ago \| parent [-]
		I have it on my 7900xtx but haven't compared it just yet. I might though. Honestly what I need is a lack of knowledge trained into the model. Maybe 27b will be better. I need it to want ground truth data, not hallucinate based on quantized distilled documents from training.