Remix.run Logo
smcleod an hour ago

I believe you mean 35B-A3B, there was no such thing as A4B. I use 27B and other models for software development, and quite a few research or similar agents. I cannot imagine a world where the old 35B-A3B model is smarter / more capable than 3.8 27B - the difference is night and day for coding at least. Where 35B-A3B was fast and felt like a Haiku model, 27B feels like a strong Sonnet when given the right tools. I don't use turbo quant so can't comment on that, but with the A3B model you're using you probably won't get much from using MTP with small MoE models like that.

xscott 13 minutes ago | parent [-]

People over-quantize things, muck with the temperature and other settings based on superstitions or results from models they think are similar. There's lots of ways to make 3.8 27B dumber.