Could this be solved with some sort of TPM-like secure attestation that can prove you’re running on the CPU you think you are, plus encrypted memory to defeat external memory reads?

For it to work, the whole CPU would pretty much need to be a secure enclave. It puts very different requirements on the hardware than affordable high performance computing does.

Even then, many secure enclaves have been compromised by people with enough time and motivation.

that's exactly what confidential vms are