I get what you're saying. But to me he is simply being honest.
If he did not wonder to himself how he would have felt about a scraggly, hippy knocking on his front door in the evening, I would have been suspect.