So what are you going to do when (not "if") your internet connection suddenly drops?

Unless you are okay with robots crashing into things at full speed, you'll always need some kind of local control loop, so full-remote control will never happen.

Offloading the goal decision to the cloud is very much possible, but that doesn't require an ultra-low-latency connection as (due to the above) the robot should be fine with no new feedback for a second anyways.