Actually, you can't unless you meet a karma threshold. I have been on this site for 6 years and I still haven't reached it because my opinions are too controversial (apparently).

Not controversial. Just not aligned with the owners of HN.

The comments you upvote/downvote are used to calculate your "opinion similarity" to the owners. Then your upvote/downvotes are modulated by that similarity. It's a way of using other humans to amplify the opinions of the owners -- they don't have time to read all the comments, so instead they filter out those who think the same way they do and let them do it.

You can't downvote submissions, regardless your karma.