If having "an hour-long manipulative conversation" was possible, we have proof that company placed an unsupervised, error prone mechanism instead of real support.

If that "difference" is so obvious to you (and you expect it will break at some point), why don't you demand the company to notice that problem as well? And simply.. not put bogus mechanism in place, at all.

Edit: to be clear. I think company should just cancel and apologize. And then take down that bot, or put better safeguards (good luck with that).