I don't think the concerns are far fetched at all. Look at how image models spit out copyrighted stuff all the time. Midjourney has a bizarre EULA clause that if you use it to generate images that violate copyright and _they_ get sued, they can hold you liable downstream. Which is wild to me -- just don't train on things you don't own and this is not a problem!

In images it's much more _obvious_, but I think code is very likely to have similar problems. Like, websites that an LLM spits out are often very very similar. It wouldn't be shocking to me if some of the code in the training set was trained off GPL code, and there are small GPL violations all over the place.