>Why should we trust the TPM or the CPU or a YubiKey or anything, really?

You raise a good point.