The reason slavery is bad isn't because of suffering, depending on how you define suffering. There were "house slaves" that had relatively comfortable lives. Slavery is bad among other reasons because it strips away someone's humanity and completely takes away their liberty, subjecting their life to the will of someone else. It's a constant ongoing theft of a human life, a reduction of a human life to property.
I'm glad you brought up suffering, I'm realizing better now why so frequently I hear these two ideas brought together by people inadvertently finding themselves on the same side as folks minimizing slavery in attempts to argue against harming animals (by engaging in debate about moral relativism). Purely from a suffering standpoint slavery doesn't necessarily have to be "that bad."
Drawing comparisons between it and arguments against harming animals are nonsensical because we're not talking about suffering, we're talking about other things that can only possibly involve humans. Thank you for sticking around and exploring your viewpoint with me so I could understand that better.