There is a lot of mathematics that can be used to analyse language. Phonetics is basically acoustics, phonology has things like optimality theory, for morphology you can use finite state machines, syntax uses formal grammars, statistics obviously plays a big role in certain areas, etc.

Both in the case of music and linguistics, there are people who argue (probably not wholly without merit) that looking at the mathematics too much is missing the point.

A refusal to acknowledge such integral parts of these systems as semantics renders that line of argument wholly irrelevant. No good faith discussion cannot be conducted without participants who already understand the meaning behind symbols.