Can we somehow make them work with 1 TB PCIes so we can churn through way more data?

You are able to use GPU Direct Storage to communicate between the GPU and PCIE storage devices. It's nice, but it's not typically as performant as one would like, in comparison to the onboard memory.

https://docs.nvidia.com/gpudirect-storage/

https://github.com/microsoft/DirectStorage/tree/main