The downvoters are their own mysterious nonverbal community. Most of the time I don't see them as part of the conversation.