> but rather that they're abused by psychiatrists

Well that but also they have poorly understood long term effects even after being discontinued (in some people, not others) and they don't work for everyone. The latter is probably most of the reason they get hated on. I don't recall the source but a given antidepressant only works for something like 1/3 or less of the population. So take a person not in a great place emotionally, who is also statistically not in a great place in life overall, subject them to an insufferable bureaucratic process, give them a drug that doesn't end up working for them, add in some pretty wild side effects, sprinkle on a few long term effects that persist after they discontinue the thing that didn't work to begin with, and of course you end up with a bad reputation.

The tl;dr is that our understanding of the brain and mood disorders kind of sucks.