Remix.run Logo
▲ peterbell_nyc 2 hours ago

This assumes material switching costs. I use frontier max plans for now but have fail over models for each class of model action with tuned context so I can keep the performance when I lose the max plan subsidy at a lower price that staying with the frontier labs for certain classes of work.

I see minimal downside in using closed weight models as long as they're behind a router, you have good prod sourced evals and you have fail over to other models configured and tested (running either dark or active:active) with real prod workloads.

▲munk-a 2 hours ago | parent [-]

If you want relatively repeatable and predictable responses (even within the reality of being non-deterministic) then making sure you'll have continued access to a model to replay responses for debugging and tuning can be important.