Yes that's true, many classical civilizations saw work as a role for slaves. The ideal for classical elites wasn't to work, it was to have enough slaves that you didn't have to work yourself.

Aristotle in particular writes about this in his defense for the institution of slavery.