> training on multiple data centers' worth of GPUs scattered around the world

...so stop scattering things across many data centers.

> iteration speeds

Which can be slowed down for the sake of not breaking the law.

> it likely needs to happen on a daily basis or multiple times a day.

Keep it inside the airgap until you are ready to move the model into production. What's the problem here?

So we just need to build the biggest datacenter in the world in downtown San Francisco. Problem solved!

	▲	betterunix2 an hour ago \| parent [-]
		Or lay a cable to wherever the data center can reasonably be built? A leased line would be an acceptable way to do this, given that the endpoints do not really have any way directly to interact with the service provider network. It is not as though this is some uniquely difficult problem, it is just a question of putting in the effort.