I think it's time to update your expectations. Hardware security modules have been around for quite a while. You shouldn't assume that your chosen password is the only input to the key derivation, or even that the raw key itself will always be obtainable.

I never assumed that my chosen password was the only input to the key derivation. Just that my chosen password is at least one of the inputs to the key derivation. LUKS has been around for quite a while.