HSMs are designed to protect encryption keys from everyone including the manufacturer. Signal trusts them for their encryption features. It's the best security possible for E2EE backups with passcode recovery, and Apple does it too for the subset of data that they do real E2EE backups on, like Keychain passwords. Characterizing using an HSM to implement E2EE securely as "not any better than" just giving up on E2EE for messages backups is ridiculous.

The HSMs that Signal and Apple are using are on-device though. Yes you still have to trust Signal / Apple to not exfil your key matter once decrypted by the HSM, but I submit that that is materially better than having the HSMs be hosted in a datacenter.