Models are not deterministic, but they do have a flavor. When that flavor changes it can change the nature of output in a way that is undesirable.

Sort of like shooting a rifle - where the bullets hit is (to some order of magnitude, no philosophizing please) non-deterministic, but different very similar rifles will group differently and need to be appropriately adjusted to hit anything.

▲

nolok an hour ago | parent | next [-]

Yeah but then it doesn't matter if it's "the model changed to another one" or "the model changed but it's the same name". Point is, you're not hosting it, as far as you know it can change at any moment, build around that idea. Is that great no, is that ideal no, that's why I self host (I include actually renting online the capacity and hosting the model myself on it).

▲

switchbak 20 minutes ago | parent [-]

You've added a similar response like 5 times, but I don't see you adding more information.

Yes - model hosts can do nasty things to you aside from changing the underlying model. That doesn't mean it's cool to have them change the model automatically.

Yes, it would be preferable to have complete control over your model serving, and no - not everyone is in a position to do that themselves.

	▲	nolok 4 minutes ago \| parent [-]
		[dead]

▲

tomrod 2 hours ago | parent | prev [-]

This is a great way to think of it, idiotsecant.

That flavor profile is known -- it's typical behavioral distribution is somewhat understood (and, often, common failure modes addressed). If JSON breaks about 20% of the time, and that drops for 2% or blows up to 90%, it can drive all sorts of issues (not the least, costs for retries).