| ▲ | zozbot234 4 hours ago | |
It's a MoE model so I'd assume a cheaper MBP would simply result in some experts staying on CPU? And those would still have a sizeable fraction of the unified memory bandwidth available. | ||
| ▲ | pitched 3 hours ago | parent [-] | |
I haven’t tried this myself yet but you would still need enough non-vram ram available to the cpu to offload to cpu, right? This is a fully novice question, I have not ever tried it. | ||