It’s a solved problem in the VM world too. Memory ballooning is a technique where a driver inside the VM kernel cooperates with the hypervisor to return memory back to the host by appearing to consume the memory from the VM. And disk access is even easier; just present a network filesystem to the VM.

The network file system to host is usually pretty slow no? That was my impression.

As for memory ballooning, the main issue with it is that it (generally) only gets triggered when the host runs out of memory.

For a host which is only running VMs, this is fine. But for the typical consumer host it becomes cumbersome as you still need to give the VM a giant memory block and hope that your VM of choice is good enough to free on time. It's also uncoordinated. When swapping needs to happen, if the VM was using the host for allocation the host could much more efficiently decide what needs to go into swap.

And if the host was in charge of both the memory and file system, then things like a system cache could be done more efficiently on top of all that.