True, but it's turtles all the way down. There is lots of non-reprogramable firmware in what you call "hardware". The recent article here pointed out the 8087 (an old floating point co-processor) had so much firmware (for the time) Intel had to use a special type of transistor to make it fit. Modern CPU's have many such tiny CPU's doing little jobs here and there. I'm being you didn't even know they exist. They not only exist, they also have a firmware programmed into ROM's you can never change. The bottom line is you have to trust the manufacturer of the silicon, and that isn't much different to trusting someone else who loaded firmware into the device.

The fact that there is always something you must trust in a device, as opposed to being able to prove it's trustworthy to yourself by just looking at it is so well known it has a name: is called the root of trust.

The interesting thing is it can ensure the root of trust the only thing you need to trust. The ability to do that makes your statement factually wrong. In fact it's drop dead simple. The root of trust only need let you read all firmware you loaded back, so you can verify it is what you would have loaded yourself. TPM's and secure boot are built around doing just that. Secure boot is how the banks and whoever else know you are running a copy of Android produced by Google.

A compromise; if the manufacturer has a way to reprogram them, then the users should be able to as well.