As a general rule of thumb in sites like Reddit and HN - the quality of votes is significantly lower than the quality of comments. This is because it takes much more effort to comment, so there is a selection bias.

I'm not convinced that downvotes add much value. They should be a "this is irrelevant/spam" button but in practice they seem to be used as a "dislike" button to enforce groupthink.