| ▲ | ismailmaj 4 hours ago | |
1.5 TB/s 1.3 PFLOPS with no fast multi-GPU, it's a consumer card that pretends to be a server card. It would work great on Qwen3.8-Flash-Next or GLM-5.3 as single user decode only hits a few experts, anything with concurrent users and you'd be sitting for a while. | ||
| ▲ | minraws 4 hours ago | parent [-] | |
That's the point I believe fast local 2-4 user service for cheap on very large models. You have to pretend it's for servers to keep investors happy, it would be banks and similar with in house servers that have to run local large models buying these for experiments and cybersecurity. Likely. | ||