It depends on the ORM. I've definitely seen a couple (popular, even) where it's worse than nothing at all, if you're comfortable in SQL. But most of them I'd agree, they're handy.

I think some of the blanket "just don't" comes from people who've had to onboard to projects written by teams that didn't understand SQL, but did (sort of) know how to use an ORM, and blame the ORM for allowing those teams to commit their atrocities. But that doesn't make an ORM a bad thing in capable hands.

Ah yeah, my experience is with the Django ORM, with which I've had nothing but good experiences with.