In the term "conman" the confidence in question is that of the mark, not the perpetrator.
Isn't confidence referring to the alternate definition of trust, as in "taking you into his confidence"?
I think if you used that definition you could equally say "it is the mark that is taking the conman into [the mark's] confidence"