How does that work? None of the current AI models can re-train on the fly. How would the inference engine even know if it's a case of new information that needs to be fed back, or just a user that's not following instructions correctly?

This is correct. What I meant to say was that in due course, re-training on the fly will become a norm. Even without on the fly re-training we are looking at a small delta.