This is precisely why we need more projects like this https://github.com/bigscience-workshop/petals

There's no fundamental reason why models couldn't be developed and trained using community efforts. It might not be as fast and efficient, but it's definitely possible.

See the recent development of DiLoCo at Nous Research and Prime Intellect.