It depends on the time of day. I have noticed that certain posts of mine are getting downvotes earlier in the day (when there are more Europeans) and upvotes later (when the Americans are in the majority). Other posts are getting upvotes early and downvotes later.

And it's the main reason why I rarely write comments on HN. Everything is so political. If people don't agree with you, they downvote you. Your karma only reflects the popularity of your views. If you're writing something that is controversial, it's likely to lower your karma, if one side can downvote it before the other side sees it.