Keeping the key in the same room as the padlock only protects against casual drive theft and secure disposal.

Personally I'm more worried about someone stealing the entire server or a local threat actor.

Sure, keep TPM to help with boot integrity, maybe even a factor for unlock, but things like Clevis+Tang (or Bitlock Network Unlock for our windows brethren) is essential in my opinion.

TPM locking is for ensuring the disk isn't removed from your machine. It's technically possible that someone could tap the hardware while the disk is still in your machine, but otherwise they're stuck contending with whatever other security setup you have on your machine.

The TPM locked disk encryption is more like embedding your safe in concrete with deep foundations. It doesn't affect the thickness or quality of your safe.