If I have a disk image or access to the physical drive, it's trivial. This means they can no longer be considered encrypted at rest.

If you're on prem or able to manipulate the machine into an OS of your choosing, yes. But with purely remote access to a device the disk is pretty decently secured (even if Window's ACLs are nightmareishly convoluted).

If your computer storage is not fully encrypted you have bigger worries than the swap file.