Secondhand smoke is toxic and physically damages your body and enters your bloodstream.

Someone playing music is annoying and does not physically harm you in any way.

These are not remotely the same thing. There is a clear bright line between them.

Unwanted noise affect stress level which has an impact in health.

It is not just harmless annoyance and not everyone is equal in that regard.

It stresses me out and distracts me from what I'm doing. You have no right to do that and I will ruin your day if you try.