| ▲ | jimbokun 18 hours ago | |||||||||||||||||||||||||
Except it’s very difficult for home users to afford the GPUs needed to run them with acceptable performance. | ||||||||||||||||||||||||||
| ▲ | notenchange 15 hours ago | parent [-] | |||||||||||||||||||||||||
I have real world data here. On a 5060 Ti (16GB) with 64GB RAM, I’m getting ~65 tok/s on decoding and ~450 tok/s prefil for Qwen 3.8 Flash Next. 256K context. This is totally suitable (speed wise) for coding. This is not some top of the line hardware (might be now, but it should be run of the mill for people that built their system <1.5 years ago). | ||||||||||||||||||||||||||
| ||||||||||||||||||||||||||