If you mean nobody is good at something, just say that.

Saying most people aren't good at it DOES imply that some are good at it.