One of the very first things the government used it for when they got their hands on it was for targeting missiles, and then they promptly bombed a school full of kids. It's like Idiocracy meets Terminator.

human repeats bad idea.

This is such a ridiculous take I hear all the time. The shitty ethics of some people do not represent the ethics of an entire profession. Most programmers are not automating people out of jobs. I've been professionally employed as a software developer for 20 years and I don't think a single thing I've ever written has ever replaced anyone's job. Hopefully sometimes it improves their job.

You sound like a webdev, e.g. the "let's just install 1000s of unvetted external dependencies" is nowhere else nearly as extreme as in web development land.

I don't think that's accurate. Until recently the code we rely on wasn't blindly trusted. People wrote and QA'd it. We may not have reviewed it personally, but that's one of the functions we outsource to software maintainers or "manufacturers".

I'm not "blindly trusting" code on my computing devices. I'm trusting the vendors / maintainers to do their job.

Until very recently the norm has been that the vast majority of code had human eyes and hands on it.

Edit:

The owners of those human eyes and hands had some type of accountability (either reputationally, in the case of free/open-source software, or occupationally, in the case of proprietary software).

The LLM has no accountability as to the output it generates.

The companies who make the LLMs also seem to have very little accountability, too. We've assumed a "blame the victim" stance when people use LLM-generated output in some inappropriate ways (legal briefs with "hallucinated" citations, articles "written" by LLMs). Whether that's the right location for accountability to be placed isn't for me to say, but that seems to be how it is.

I'm not sure that we're applying accountability to LLM-generated code in the same way we are for, say, the LLM-generated legal brief.

Or they're self conscious about their use of AI so they take a maximalist stance to compensate.

I don't think the concerns are far fetched at all. Look at how image models spit out copyrighted stuff all the time. Midjourney has a bizarre EULA clause that if you use it to generate images that violate copyright and _they_ get sued, they can hold you liable downstream. Which is wild to me -- just don't train on things you don't own and this is not a problem!

In images it's much more _obvious_, but I think code is very likely to have similar problems. Like, websites that an LLM spits out are often very very similar. It wouldn't be shocking to me if some of the code in the training set was trained off GPL code, and there are small GPL violations all over the place.

The stance may be reasonable, but the reasoning is not clear to me and the limit does sesm to suggest copyright concerns that are, as you say, far fetched.

They still allow LLM-generated code for tests, LLM-assisted debugging, bug discovery and code reviews. It doesn't seem so.

It's far-fetched that the GNU project is authentically concerned about software licensing?

I guess that's where the rage comes from. People that believe a policy is without reason.

In many timelines, non-copyrightable code in GPL project doesn’t restrict the user’s freedom, since an LLM can easily reimplement the functionality clean-room. AI has brought levels of freedom to users that GNU could never do. Practical levels of freedom, leading to real, positive outcomes. Software is finally free in the real world.

Not to mention the moral concern.

IP law (like a lot of other things) has been skewed toward the interests of business, even when that conflicts with fairness or societal good. For all its flaws (IMO), the free software movement tends to be principled. Just because something is legal doesn’t mean it’s right.

+1, I also thought this was the actual concern.

Not just when it reproduces code. Since fair use doesn't exist like that in a lot of countries the question is if the training itself is a violation. Are there e.g. EU or Japanese rulings on that?

I had early chatGPT spit my own erlang code back at me WITH my comments.

To my knowledge there is not a single known case where this happened.

News back then were about intentionally prompting to output known copyrighted material.

The parent comment still stands in my opinion:

When, despite millions of developers using agentic AI already, are these lawsuits supposed to manifest?

That would be an example of, “overfitting.” It is something the AI labs are concerned about. They tweak parameters to try and avoid it.

It’s a hard balancing act to do. Give in too much randomness and you get non-sensical outputs that are difficult to align. Fit too closely to the training data and the model regurgitates the training data.

And oh, what’s that copyrighted material we never made any agreement to use doing in there?

This doesn't really happen by accident, especially where it's significant enough to constitute copyright infringement. And to my knowledge, there hasn't been a single lawsuit over this in the entire history of LLM-assisted coding.

There is concern, but the more I think about it, the less convinced I am. LLMs might produce a snippet, but copyright requires more than that. Last I checked (I'm not a lawyer, this was far enough back that the law has had major changes, and different countries have different laws! - read this with plenty of salt) if you start with copyright work and replace all but 10% that final amount just disappears since there isn't enough original to claim copyright anymore. You can never claim copyright over 'printf("Hello World");' - it is too generic. Most of what I've seen an LLM produce is likewise generic enough that even if it is a copy, it wouldn't be a copyrightable snippet.

Again, it would require an extremely unlikely set of circumstances.

1) All of GCC would have to be LLM generated. If some parts are not, it's sufficient to prevent the re-licensing.

2) That someone would have to prove that all the GCC code was in fact LLM generated. Good luck doing that.

3) A court would have to decide that all the LLM-generated code does in fact fall under public domain, because it involved insufficient human input.

> Someone re-licenses GCC under a non-GPL license.

A pointless act since code is now free. The GPL exists to ensure code freedom in an era when code was expensive. Yes I'm aware that the meaning of "free" is a bit conflated here, but the point stands.