They’re not in swap if those processes wake up to do things

Yeah but they mostly barely do, and only the memory they actually access gets used, as opposed to everything they've allocated. You can observe the actual aggregate usage in Activity Monitor. This is why it's no problem at all to run something actively using 10-12 GB of memory on a 16 GB Mac.