I think the right way to frame it is that the particular format of a site doesn't necessarily dictate whether or not something is "social media". It's important to look at how people actually use the site[0], as well as whether the voting/ranking/whatever system looks more like a popularity contest or objective moderation. It usually won't be 100% of either of those, but it will certainly lean one way or the other. I think HN leans toward objective moderation much more than Reddit does, even though HN's moderation is certainly not fully objective.

[0] This use can be heavily influenced by how the owners of the site push things, e.g. HN's guidelines and in-house moderation decisions vs. Facebook's algorithmic news feed that chases user engagement above all else.