Excuse my ignorance. Could one just say, "One expert is all I can handle" and strip the others from the model?

I have a similar question and I’m inferring the answer is no - look at the cache hit rate of 23% for the 128GB M5 Max. I had previously assumed that the 40B active meant that a set of layers was chosen as THE expert for a given prompt and generation was then limited to those layers until complete. But in that case you’d have expected the expert caching to have a super high hit rate once you had enough RAM to hold an entire expert’s worth of layers.

You could (e.g. by replacing residual-dependent expert routing with hardcoded logic), but quality will suffer dramatically. It’s far better to use a similar-sized dense model then.