It would raise the same kind of alert for me if someone used wget to download a binary executable instead of a shell script.

The issue is not the specific form in which code is executed on your machine, but rather who is allowed by you to run code on your computer.

I don't trust arbitrary websites from the Internet, especially when they are not cryptographically protected against malicious tampering.

However, I do trust, for instance, the Debian maintainers, as I believe they have thoroughly vetted and tested the executables they distribute, with a cryptographic signature, to millions of users worldwide.