I drop my phone occasionally. I don't think there is much evidence that I am uniquely non-clumsy. Phones aren't that fragile, particularly these days.

Some of it may be differences in skin. I have fairly dry skin and an aluminum phone is slippery as hell. I put mine in a case not for protection from drops - I almost never drop it - but to prevent those drops by giving me a little grip. YMMV