You have to have a minimum amount of karma here to get a down vote button. I think it's 500.
Correct. And to add: it only applies to comments; non-moderators cannot downvote stories