> I extremely quickly dismiss them in the most polite, but firm, way possible.

And I think that's the answer; people who don't want to talk will simply tell you! And everyone carries on.

But they said it's rude just to speak to them ... which is a factually erroneous characterization.