Remix.run Logo
▲ flatline 22 minutes ago

I’m getting ~6.7 tok/s out of it on an a laptop RTX 4060 with 8GB VRAM — over twice the speed I can get out of qwen, and arguably fast enough to turn it loose on real work. For a minimal hardware setup, this is easily the best speed for performance I’ve found in a model. The nemotron models are the next best, they are much faster, much more verbose, and much less accurate.

▲6_7 14 minutes ago | parent [-]

I'm so sorry you're getting how many tok/s?

▲flatline 3 minutes ago | parent [-]

Last gen hardware before local AI was even a consideration. Things have actually gotten measurably better.