I think the future should be efficient quantization which can make model with the same level as Qwen 3.8 27b now accessible from our laptop.

▲

Moto7451 2 hours ago | parent [-]

This is mostly a memory bandwidth issue. Qwen 3.8 27b and the related Bonsai 2 27B run on my M5 Max laptop with 36GB of ram. It’s a little slow due to my specific model having lower memory bandwidth than higher end models with more ram and cores. MoE models perform better here and expert streaming engines can allow larger than memory models to run.

	▲	nout 2 hours ago \| parent [-]
		Multiple teams and efforts are now making Qwen 3.8 27b work much faster on Mac and CUDA and the inference speed is doubling almost every week. There's now ongoing competition about who will make it work faster with exactly the same byte output at https://mlx.fast . TensorFold is the project to follow.