I've been building multiple products with LLMs, and they are in fact interchangeable for the most part.

In fact, most benchmarks show this! Most benchmarks have similar performance for the same classes of models.

On top of this, there are tools like open router, or even the openai SDK which trivially allows you to swap endpoints for the LLM!

If you're using the agents SDK from openai or something, then yeah it's not interchangeable but that's you doing it wrong