This does not appear to be true. Six months ago I created a small programming language. I had LLMs write hundreds of small programs in the language, using the parser, interpreter, and my spec as a guide for the language. The vast majority of these programs were either very close or exactly what I wanted. No prior source existed for the programming language because I created it whole cloth days earlier.

▲

jazzyjackson a day ago | parent | next [-]

Obviously you accidentally recreated a language from the 70s :P

(I created a template language for JSON and added branching and conditionals and realized I had a whole programming language. Really proud of my originality until i was reading Ted Nelson's Computer Lib/Dream Machines and found out I reinvented TRAC, and to some extent, XSLT. Anyway LLMs are very good at reasoning about it because it can be constrained by a JSON schema. People who think LLMs only regurgitate haven't given it a fair shot)

	▲	zahlman a day ago \| parent [-]
		FWIW, I think a JSON-based XSLT-like thing sounds far more enjoyable to use than actual XSLT, so I'd encourage you to show it off.

▲

fpoling a day ago | parent | prev [-]

Languages with reasonable semantics are rather similar and LLMs are good at detecting that and adapting from other languages.

▲

pertymcpert 20 hours ago | parent [-]

Sounds like creativity and intelligence to me.

	▲	tatjam 11 hours ago \| parent [-]
		I think the key is that the LLM is having no trouble mapping from one "embedding" of the language to another (the task they are best performers at!), and that appears extremely intelligent to us humans, but certainly is not all there's to intelligence. But just take a look at how LLMs struggle to handle dynamical, complex systems such as the "vending machine" paper published some time ago. Those kind of tasks, which we humans tend to think of as "less intelligent" than say, converting human language to a C++ implementation, seem to have some kind of higher (or at least, different) complexity than the embedding mapping done by LLMs. Maybe that's what we typically refer to as creativity? And if so, modern LLMs certainly struggle with that! Quite sci-fi that we have created a "mind" so alien we struggle to even agree on the word to define what it's doing :)