> A.I. doesn't steal code, and just reproduces the ideas in the code

Then they don't need to train on github, no? Why not release a new model trained from Knuth's Art of Programming, Cormen's Introduction to Algorithms and the C specification.

Feel free to throw in any other published literature related to STEM, but stick to the code samples from the books.

I'm certain it'll be able to change the color of a CSS button, right?

What you said doesn't disagree with what the parent said.

LLM can be trained on a code and at the same time reproduce the core ideas. That's what LLMs do after all - they convert the training data into their own internal models and representations, and then reproduce the ideas.

Sure, some things/patterns, that were repeated multiple times, LLMs will tend to repeat verbatim as well, but that's not that big of a problem.

As a person who invented a few algorithms on my own I absolutely love LLMs and I don't mind them being trained on my work, but yeah - I've been way less likely to publish open source over the last year. In the past, if some of my stuff got traction, the credit was close to automatic (early adopters credited or at least knew where they got it from). Nowadays, LLMs will train on these ideas, rewrite them, and give no credit.

Still, I prefer this to having no LLMs at all.

> but stick to the code samples from the books. > I'm certain it'll be able to change the color of a CSS button, right?

A good enough LLM will just decompile a browser, figure out CSS spec from it, and yes - figure out how to change the color of a CSS button from first principles. There is no point to do this with CSS, but with other things it's now easier to just dig through sorces or direct bytecode than to bother checking docs.

> Then they don't need to train on github, no? Why not release a new model trained from Knuth's Art of Programming, Cormen's Introduction to Algorithms and the C specification.

Because they're really really stupid and only make up for this by being really really stupid really really fast.

This has been ruled, by actual courts, to not be "stealing" (not even in the "you wouldn't steal a car, piracy is theft" sense that film and music studios campaigned on).

The last I heard was the "Chinchilla" scaling law was ~20 training tokens per parameter. Humans are, if you'll excuse a very hand-waving Fermi estimate, 100,000 times more data-efficient at learning stuff (it's really hard to tell given we're visual creatures that happen to speak, while LLMs are text-based things that happen to see).

To be fair, you also don't restrict yourself to those texts either. You read news, you use other programs, you look at websites and so on. And while the norms vary per field, things aren't really reinvented from scratch. The standard FPS controls aren't reinvented for every shooter game. The standard website layouts aren't reinvented for every website. The standard command line behaviour isn't reinvented for every CLI program and so on.

> "Why not release a new model trained from Knuth's Art of Programming, Cormen's Introduction to Algorithms and the C specification."

What makes you think that wouldn't work? I think a lot of the hype around AI is vastly overblown but that seems to be well within the scope of what they can be expanded to do in the not too distant future. AlphaGo was trained through self-play reinforcement learning IIRC and I don't really see a reason that some sort of equivalent couldn't be done for generating code starting with textbooks and access to a Linux CLI as a reference. It would be an interesting experiment at least.

I'm sorry, but what are you trying to say here? There are books that teach you how to change the color of a CSS button...