The top LLM companies could fund the purchase of the training material. One LLM thinks that Models like: Mistral AI, Stability AI, university labs, independent researchers might never catch up because training data becomes a gated asset. That sounds like a very reasonable assessment.

So what's your preference?

My preference is that if you need to use terabytes of data to train an LLM, that data should be used according to its copyright, and with the consent of the copyright holder, not just hoovered up from wherever you can find just a few bytes more data