> It’s weird that people don’t seem to get this. LLMs are the most easily switched-out technology I’ve ever used as an engineer. The input is literally just free-form text.

It depends, if you're automating business processes (honestly probably where the money is), then you'll need a really comprehensive set of evals and moniotoring to ensure stuff doesn't break if/when you change providers.

If you have none of that, then switching will be very expensive (but it was ever thus).

▲

72deluxe 31 minutes ago | parent [-]

That just means the business automation is not deterministic, and highlights the issue with LLMs. The solution is not to switch providers but instead make the LLMs generate something that is deterministic given the same input.

	▲	disgruntledphd2 11 minutes ago \| parent [-]
		> That just means the business automation is not deterministic, and highlights the issue with LLMs. The solution is not to switch providers but instead make the LLMs generate something that is deterministic given the same input. I'm thinking of stuff that would typically be outsourced (like compliance stuff for identity verification et al), where it's really really hard to have deterministic rules. LLMs are definitely a step up in dealing with fuzzy inputs, and enables you to mostly automate stuff that really wasn't feasible before. But you do need really good evals and monitoring to make this work effectively.