| ▲ | aphroz 4 hours ago |
| I think not much can run without a dedicated GPU |
|
| ▲ | madduci 4 hours ago | parent | next [-] |
| Till now I was using successfully Qwen 3.5 and Gemma 4 at a reasonable speed |
| |
| ▲ | pyrale 3 hours ago | parent | next [-] | | There is no way you would run a dense 27b model on that spec. I ran 3.6 27b on a 64gb ram, 24 gb vram, and it felt like the lower limit for this model with a decent context window. If you want a better experience, maybe wait for either a moe model (like 3.6 35b A3) or a model with less parameters (like 9b). Qwen has been releasing those in the past, so maybe we’ll have them for 3.8 too. | |
| ▲ | mobelkh 4 hours ago | parent | prev [-] | | were you running the MoE models? those perform better speed wise |
|
|
| ▲ | noduerme 2 hours ago | parent | prev [-] |
| What's the story with Mac laptops? Worth a try? |
| |
| ▲ | cyberrock 43 minutes ago | parent | next [-] | | Dense ones like this are more bandwidth-hungry, so you want to try MoE ones like Qwen3.6-35B-A3B (35 Billion params but only 3 Billion Active) or Gemma 4. Unfortunately it seems like we might not be getting a 3.8 MoE. | |
| ▲ | selcuka 2 hours ago | parent | prev | next [-] | | The author tested in on an M5 laptop too: > It feels pretty slow on both the M5 Mac and the DGX Spark. | |
| ▲ | rawland 2 hours ago | parent | prev [-] | | Yes. mtplx runs it at 25 tok/sec on a M4 Max with 48GB RAM. |
|
