Isn’t the client-side case something like “the banking app you’re entering your account password into is the binary the bank created and not a compromised binary that will drain your bank account”?

No, this would just require a publicly verifyable signature of the software, and the user would just choose to have their operating system verify it. No remote attestation or other hand-over-your-controls necessary.