We don’t know “what we are seeing” because we are looking from the outside. That’s my point. We can see a chat bot and we can see bad behavior and there are clearly a lot of assumptions that the problem is that someone gave the bot a set of general tools and a prompt and it went off the rails. And that is a possible scenario. It’s also possible that they stuck a dumb chatbot in front of an existing automated account reclamation flow that worked exactly this way but no one noticed.

Do we actually know that a human was in the loop before and that the human judgement was replaced by an LLM? Or is that pure speculation?

I have certainly seen account reclamation flows that allowed providing a new email address (but usually with better safeguards).