I find that it's like telling twins apart. If you don't know them they may be very similar, but once you spend some time with them it can become very obvious as to who is who.

We all have our own rules that we've built up over our lifetime and some of us have definitely spent more time watching people and/or are just better at the subtle clues.

But having said that, there are often people that just break all the conventional rules or just doesn't trigger any of the things that you are looking for. Like most things you can probably draw a bell curve where we can easily spot the people in the middle, but then it becomes geometrically more difficult as you move to the sides.