Remix.run Logo
▲ ffsm8 7 hours ago

I don't work for anthropic, so I don't know how they've implemented their MOE... But at least with the ones I know you cannot easily "attach" another expert to the model like that. Maybe they do have a special sauce that let's them do so. If so, it may be viable but it's effect would be a lot less effective then the full version this discussion was started on.