From what I have seen that is not the case. The AirPods nor connected iPhones are interacting with the other person’s at all.