It really depends on what you're doing, but most LLM usage and agentic runs are pretty interchangeable in my experience, and it's usually trivial to switch.

If anything, you're better off supporting multiple LLMs as backup because most model providers have been so inconsistent with working all the time

just use your agents to do the migration, that's what it's good at.

Exactly, as in, really, will they? Where and at what price, especially across an actual enterprise that needs to deploy them to lots of devs? There's much more than just the actual model.

Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally with a bunch of open weight models.

I can run super tiny models on my 8GB NVIDIA card. They all suck (I have to use <=~5GB models if I want "usable" ~250k context that doesn't need to use system RAM and CPU (which makes things super slow).

I've also tried a GLM 4.7-flash, which even though it's super slow (in comparison) with ~250k context and it just doesn't cut it vs. the Claude Sonnet or Opus I get to use at work. All the while these are all touted as "totally usable, Claude/ChatGPT killer!" replacements.

It's just not "there" with tool use or building software for that matter. Like, just a simple Claude "web search" fails with it. So I asked it to build itself its own "web search" functionality and it just couldn't. It made so many mistakes its just not funny any more. And it couldn't recover from them either. I retried a few times (as I didn't have python installed and it wanted to implement it using that - this happens to be new system - never mind other attempts). I spent as much time doing this (and failing) as I spent building an actual full feature at work last week w/ Sonnet.

If it can't build itself a simple web search to .md file tool/skill, how am I supposed to trust this with actual coding? I'm used to being able to point Claude at our large code base and essentially work with it like a junior doing my bidding. Maybe 5.2 is a killer game changer vs. what I was able to try out (if slowly) but you really have to show me to convince me at this point. And not with synthetic benchmarks. In those, all of the models I tried are supposedly super awesome.

This is a meme and massively over-complicating what is ultimately quite simple.

Agents will make all of these migrations trivial. I expect margin collapse across a lot of tech darlings.

How could switching out a stateful service be easier than stateless?

This is just another example of the bitter lesson. In a year a model will come out that will make none of these model specific optimizations you made matter.

Most large orgs do not need to train end users. They just need to add glm-5.2 to their router and their in house harness will pick it up. Then slowly limit usage on anthropic models and people will swap willingly. It's a simple /model command in every harness.

The inertia is legal and financial. People are paying Anthropic through AWS accounts because the simple reason of not dealing making new contract and legal agreements is enough of reason of the inertia.

But, eventually, I’m quite sure that AWS will also provide open models with those contracts without any inertia. Copilot is already offering Kimi.

My company has a deal with Devin and they provide new models all the time, and open models are becoming the most used ones by our internal metrics, especially because the company is very worried about cost.

What "training" do you have to do to get a professional developer to switch LLMs or harnesses? Its literally just download the other one, point it to your code base and start typing into that text box instead of the other one.

What would "training" even entail for that? As far as I can tell, using these tools directly is basically identical in terms of what you need to know. If you happen to have a bunch of custom configurations, maybe you need to invest some time into porting them, but it's not clear to me why you think that anyone would need to be trained if they spent months using one tool and then suddenly had t switch to the other.

Enterprises switched from openai to anthropic this year - anthropic overtook openai for the first time. I don't see why they wouldn't switch again.

There's barely any moat. All the data is with connectors, memory is near useless

Plugins and skills are completely trivial to move and most work with any model. What is not trivial are Anthropic's new managed agents vendor lock-in offering.

Is it? I switched to Kiro and it's essentially identical.. well a bit better because you get a better idea of what the harness is doing, but otherwise identical.

I made my own. It's true I don't want to switch to another harness now.

But switching models is just a command.

The irony is that Claude will help you migrate away from itself

AI is possibly the first product in history that will eagerly help you replace it with one of its competitors.

Most of the AI companies have OpenAi compatible API's, so you just get a subscription from another provider and change the URL that your LLM Agent Harness uses to talk to the AI.

I use OpenRoutet which lets you switch between providers (Anthropic, ChatGPT, Z-AI) whenever you want. Sometimes I'll have two different models from different providers evaluate each other's answers.