> Pre-training is, actually, our collective gift

I feel like this wording isn't great when there are many impactful open source programmers who have explicitly stated that they don't want their code used to train these models and licensed their work in a world where LLMs didn't exist. It wasn't their "gift", it was unwillingly taken from them.

> I'm a programmer, and I use automatic programming. The code I generate in this way is mine. My code, my output, my production. I, and you, can be proud.

I've seen LLMs generate code that I have immediately recognized as being copied a from a book or technical blog post I've read before (e.g. exact same semantics, very similar comment structure and variable names). Even if not legally required, crediting where you got ideas and code from is the least you can do. While LLMs just launder code as completely your own.

▲

jll29 an hour ago | parent | next [-]

> there are many impactful open source programmers who have explicitly stated that they don't want their code used to train these models and licensed their work in a world where LLMs didn't exist. It wasn't their "gift", it was unwillingly taken from them.

There are subtle legal differences between "free open source" licensing and putting things in the public domain. If you use an open source license, you could forbid LLM training (in licensing law, contrary to all other areas of law, anything that is not granted to licensees is forbidden). Then you can take the big guys (MSFT, Meta, OpenAI, Google) to court if you can demonstrate they violated your terms.

If you place your software into the public domain, any use is fair, including ways to exploit the code or its derivatives not invented at the time of release.

Curiosly, doesn't the GPL even imply that if you pre-tain an LLM with GPLed code and use it to generate code (Claude Code etc.) that all generated code -- as derived intellectual property that it clearly is -- must also be open sourced as per GPL terms? (It would seem in the spirit of the licensors.) Haven't seen this raised or discussed anywhere yet.

	▲	zahlman 29 minutes ago \| parent \| next [-]
		> If you use an open source license, you could forbid LLM training Established OSS licenses are all from before anyone imagined that LLMs would come into existence, let alone train on and then generate code. Discrimination on purpose is counter to OSI principles (https://opensource.org/osd): > 6. No Discrimination Against Fields of Endeavor > The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research. The GPL argument you describe hinges on making the legal case that LLMs produce "derived works". When the output can't be clearly traced to source input (even the system itself doesn't know how) it becomes rather difficult to argue that in court.
	▲	singpolyma3 29 minutes ago \| parent \| prev [-]
		You pre suppose that output is derive work (not a given) and that training is not fair use (also not a given). If the courts decide to apply the law as you assume the AI companies are all dead. But they are all betting that's not going to be the case. And since so much of the industry is taking the bet with them... The courts will take that into account

▲

yuvadam 3 hours ago | parent | prev | next [-]

I don't think it's possible to separate any open source contribution from the ones that came before it, as we're all standing on the shoulders of giants. Every developer learns from their predecessors and adapts patterns and code from existing projects.

▲

jakkos 2 hours ago | parent | next [-]

If you fork an open source project and nuke the git history, that's considered to be a "dick move" because you are erasing the record of people's contributions.

LLMs are doing this on an industrial scale.

▲

OJFord 2 hours ago | parent [-]

I don't really understand how that isn't allowed/disallowed simply on the basis of whether the licence permits use without attribution?

▲

FeteCommuniste an hour ago | parent [-]

The hard truth is that if you're big enough (and the original creator is small enough) you can just do whatever you want and to hell with what any license says about it.

	▲	zahlman 27 minutes ago \| parent [-]
		To my understanding, the expensive lawyers hired by the biggest people around, filtered through layers of bureaucracy and translated to software teams, still result in companies mostly avoiding GPL code.

▲

antirez 2 hours ago | parent | prev | next [-]

Exactly that. And all the books about, for instance, operating systems, totally based on the work of others: their ideas where collected and documented, the exact algorithms, and so forth. All the human culture worked this way. Moreover there is a strong pattern of the most prolific / known open source developers being NOT against the fact that their code was used for training: they can't talk for everybody but it is a signal that for many this use is within the scope of making source code available.

	▲	jakkos 2 hours ago \| parent [-]
		> their ideas where collected and documented Yeah, documented and credited. I'm not against the idea of disseminating knowledge, and even with my misgivings about LLMs, I wouldn't have said anything if this blog post was simply "LLMs are really useful". My comment was in response to you essentially saying "all the criticisms of LLMs aren't real, and you should be uncompromisingly proud about using them". > Moreover there is a strong pattern of the most prolific / known open source developers being NOT against the fact that their code was used for training I think it's easy to get "echo-chambered" by who you follow online with this, my experience has been the opposite, i don't think it's clear what the reality is.

▲

heavyset_go 2 hours ago | parent | prev | next [-]

You can say that about literally everything, yet we have robust systems for protecting intellectual property, anyway.

▲

Imustaskforhelp 2 hours ago | parent | prev [-]

> I don't think it's possible to separate any open source contribution from the ones that came before it, as we're all standing on the shoulders of giants. Every developer learns from their predecessors and adapts patterns and code from existing projects.

Yes but you can also ask the developer (wheter in libera.irc, or say if its a foss project on any foss talk, about which books and blogs they followed for code patterns & inspirations & just talk to them)

I do feel like some aspects of this are gonna get eaten away by the black box if we do spec-development imo.

▲

frizlab an hour ago | parent | prev | next [-]

> It wasn't their "gift", it was unwillingly taken from them.

Yes. Exactly. As a developer in that case I feel almost violated in my trust in “the internet.” Well it’s even worse, I did not really trust it, but did not think it could be that bad.

▲

vbezhenar 2 hours ago | parent | prev | next [-]

Intellectual property is not absolute and can be expropriated, just like any other property.

▲

p-e-w 3 hours ago | parent | prev | next [-]

> I feel like this wording isn't great when there are many impactful open source programmers who have explicitly stated that they don't want their code used to train these models

That’s been the fate of many creators since the dawn of time. Kafka explicitly stated that he wanted his works to be burned after his death. So when you’re reading about Gregor’s awkward interactions with his sister, you’re literally consuming the private thoughts of a stranger who stated plainly that he didn’t want them shared with anyone.

Yet people still talk about Kafka’s “contribution to literature” as if it were otherwise, with most never even bothering to ask themselves whether they should be reading that stuff at all.

▲

sneak 2 hours ago | parent | prev | next [-]

If you publish your code to others under permissive licenses, people using it to do things you do not want is not something being unwillingly taken from you.

You can do whatever you want with a gift. Once you release your code as free software, it is no longer yours. Your opinions about what is done with it are irrelevant.

▲

hjoutfbkfd 2 hours ago | parent | prev | next [-]

when you inplement a quick sort, do you credit Hoare in the comments?

	▲	jakkos 2 hours ago \| parent \| next [-]
		No, in the same way that I wouldn't cite Euler every time I used one of his theorems - because it's so well known that its history is well documented in countless places. However, if I was using a more recent/niche/unknown theorem, it would absolutely be considered bad practice not to cite where I got it from.
	▲	OJFord 2 hours ago \| parent \| prev \| next [-]
		If I was implementing any known (named) algorithm intentionally I think I would absolutely say so in a comment (`// here we use quick sort to...` and maybe why it's the choice) and then it's easy for someone to look up and see it's due to Hoare or whoever on Wikipedia etc.
	▲	antirez 2 hours ago \| parent \| prev [-]
		Now many will downvote you because this is an algorithm and not some code. But the reality is that programming is in large part built looking at somebody else code / techniques, internalizing them, and reproducing them again with changes. So actually it works like that for code as well.

▲

bko 2 hours ago | parent | prev [-]

I don't understand this perspective. Programmers often scoff at most other examples of intellectual property, some throwing it out all together. I remember reading Google vs Oracle where Oracle sued Google for stealing code to perform a range check, about about 9 lines long, used to check array index bounds.

I guess the difference is AI companies bad? This is transformative technology creating trillions in value and democratizing information, all subsidized by VC money. Why would anyone in open source who claims to have noble causes be against this? Because their repo will no longer get stars? Because no one will read their asinine stack overflow answer?

https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_....