Look at how many browser exploits need JS, and how many don't. The latter tend to be far more memorable too, specifically for that reason.

JS is a huge attack surface. It's better if it isn't used where it isn't actually needed.