| ▲ | zozbot234 10 hours ago | |
There's a 31B dense model in the Gemma 4 series that's obviously going to be smarter (though a whole lot slower) than the MoE 26A4B. | ||
| ▲ | the_pwner224 10 hours ago | parent [-] | |
I tried it and it was unusably slow at ~5-6 TPS. 26A4B gets close to 40 TPS which is faster than you can read, and still pretty quick with reasoning enabled. | ||