Remix.run Logo
dofm 5 hours ago

The issue is not RAM size. It's memory bandwidth!

The 12B and 31B models will be useless on a Pi 5; maybe the 12B can be persuaded to run, but it may not manage more than one token per second. It only manages 17GB/s memory bandwidth (I have seen a suggestion that the 4GB model manages more).

But yes — some sort of small reasoning-oriented model (Ornith?) seems a better candidate than Qwen 35B.

(Don't get me wrong, I think the 35B model is ace… just seems like at least an unusual choice here)