Addressed on Lex Fridman with Dario Amodei (CEO) and Amanda Eskell of Claude, they both insist the answer is no.

I interpret their explanation for “no” as follows: these are probabilistic outputs, and so given any changes, for some inputs, some outputs will be worse some of the time.

The argument goes that, given they’re probabilistic, even without changes, for some inputs, some outputs will be worse than the last time you gave it that input, some of the time.

To be fair to them, it makes sense that any change would then be met with some vocal users who are genuinely experiencing worse output, but are not generally using a worse product.