| ▲ | peterbell_nyc 2 hours ago | |
This assumes material switching costs. I use frontier max plans for now but have fail over models for each class of model action with tuned context so I can keep the performance when I lose the max plan subsidy at a lower price that staying with the frontier labs for certain classes of work. I see minimal downside in using closed weight models as long as they're behind a router, you have good prod sourced evals and you have fail over to other models configured and tested (running either dark or active:active) with real prod workloads. | ||
| ▲ | munk-a 2 hours ago | parent [-] | |
If you want relatively repeatable and predictable responses (even within the reality of being non-deterministic) then making sure you'll have continued access to a model to replay responses for debugging and tuning can be important. | ||