The whole "reproduces training data vebatim" is a red herring.

It reproduces _patterns from the training data_, sometimes including verbatim phrases.

The work (to discover those patterns, to figure out what works and what does not, to debug some obscure heisenbug and write a blog post about it, ...) was done by humans. Those humans should be compensated for their work, not owners of mega-corporations who found a loophole in copyright.