| ▲ | dvt 5 hours ago |
| Because by definition LLMs are permutation machines, not creativity machines. (My premise, which you may disagree with, is that creativity/imagination/artistry is not merely permutation.) |
|
| ▲ | fnordpiglet 5 hours ago | parent | next [-] |
| I prefer to think of it as they’re interpolation machines not extrapolation machines. They can project within the space they’re trained in, and what they produce may not be in their training corpus, but it must be implied by it. I don’t know if this is sufficient to make them too weak to create original “ideas” of this sort, but I think it is sufficient to make them incapable of original thought vs a very complex to evaluate expected thought. |
| |
|
| ▲ | lukol 5 hours ago | parent | prev | next [-] |
| This "new math" might be a recombination of things that we already know - or an obvious pattern that emerges if you take a look at things from a far enough distance - or something that can be brute-forced into existence. All things LLMs are perfectly capable of. In the end, creativity has always been a combination of chance and the application of known patterns in new contexts. |
| |
| ▲ | dvt 5 hours ago | parent [-] | | > This "new math" might be a recombination of things that we already know If you know anything about the invention of new math (analytic geometry, Calculus, etc.), you'd know how untrue this is. In fact, Calculus was extremely hand-wavy and without rigorous underpinnings until the mid 1800s. Again: more art than science. | | |
| ▲ | jfyi 4 hours ago | parent | next [-] | | Newton and Leibniz were "hand-waving"? If anything, they were fighting an uphill battle against the perception of hand-waving by their contemporaries. | | |
| ▲ | dehsge 2 hours ago | parent | next [-] | | It’s not that. Consider the definition of the limit. The idea existed for a long time. Newton/Leibniz had the idea. That idea wasn’t formally defined until 134 years later with epsilon-delta by Cauchy. That it was accepted. (I know that there were an earlier proofs) There’s even arguments that the limit existed before newton and lebnitz with Archimedes' Limits to Value of Pi. Cauchy’s deep understanding of limits also led to the creation of complex function theory. These forms of creation are hand-wavy not because they are wrong. They are hand wavy because they leverage a deep level of ‘creative-intuition’ in a subject. An intuition that a later reader may not have and will want to formalize to deepen their own understanding of the topic often leading to deeper understanding and new maths. | |
| ▲ | dvt 4 hours ago | parent | prev [-] | | > Newton and Leibniz were "hand-waving"? Yes, and it's pretty common knowledge that Calculus was (finally) formalized by Weierstrass in the early 19th century, having spent almost two centuries in mathematical limbo. Calculus was intuitive, solved a great class of problems, but its roots were very much (ironically) vibes-based. This isn't unique to Newton or Leibniz, Euler did all kinds of "illegal" things (like playing with divergent series, treating differentials as actual quantities, etc.) which worked out and solved problems, but were also not formalized until much later. | | |
| ▲ | jfyi 4 hours ago | parent | next [-] | | I think that I just take issue with the term "hand-waving" as equated to intuition. Yeah it lacked formal rigor, but they had a solid model that applied in detail to the real world. That doesn't come from just saying, "oh well, it'll work itself out". I guess if you want to call that "hand-wavy" we'll just have to disagree. | |
| ▲ | anthk 3 hours ago | parent | prev [-] | | Euclid tells me otherwise. Rules, no art, no bullshit. Rules. Humanities people somehow never get it. Is not about arithmetics. Vibe-what? Vibe-bullshit, maybe; cathedrals in Europe and such weren't built by magic. Ditto with sailing and the like. Tons of matematics and geometry there, and tons of damn axioms before even the US existed. Heck, even the Book of The Games from Alphonse X "The Wise" has both a compendia of game rules and even this https://en.wikipedia.org/wiki/Astronomical_chess
where OFC being able on geometry was mandatory at least to design the boards. On Euclid: https://en.wikipedia.org/wiki/Euclid%27s_Elements PD: Geometry has tons of grounds for calculus. Guess why. |
|
| |
| ▲ | baq 4 hours ago | parent | prev [-] | | And yet nowadays you can restate all of it using just combinations of sets of sets and some logic operators. |
|
|
|
| ▲ | nh23423fefe 5 hours ago | parent | prev | next [-] |
| god of the gaps |
| |
|
| ▲ | satvikpendem 4 hours ago | parent | prev | next [-] |
| What is creativity if not permutation? A brain has some model of the world and recombines concepts to create new concepts. |
| |
| ▲ | d3ffa 4 hours ago | parent [-] | | [flagged] | | |
| ▲ | rowanG077 4 hours ago | parent | next [-] | | This is really not an acceptable reply. How about actually engaging with the point the commenter made instead of stamping your foot and throwing a tantrum. | |
| ▲ | anthk 3 hours ago | parent | prev [-] | | Innovation it's just another word for the term 'enhanced copy'. Everything it's a copy, except for nature. |
|
|
|
| ▲ | KoolKat23 5 hours ago | parent | prev | next [-] |
| It pretty much is, otherwise it is randomness or entropy. |
|
| ▲ | lajamerr 5 hours ago | parent | prev [-] |
| LLMs by themselves are not able to but you are missing a piece here. LLMs are prompted by humans and the right query may make it think/behave in a way to create a novel solution. Then there's a third factor now with Agentic AI system loops with LLMs. Where it can research, try, experiment in its own loop that's tied to the real world for feedback. Agentic + LLM + Initial Human Prompter by definition can have it experiment outside of its domain of expertise. So that's extending the "LLM can't create novel ideas" but I don't think anyone can disagree the three elements above are enough ingredients for an AI to come up with novel ideas. |
| |
| ▲ | awesome_dude 4 hours ago | parent [-] | | You're proving the GP's argument - LLMs aren't creative you say as much, it's the driving that is the creative force | | |
| ▲ | lajamerr 4 hours ago | parent | next [-] | | You can tell an agentic system. "Go and find a novel area of math that has unresolved answers and solve it mathematically with verified properties in LEAN. Verify before you start working on a problem that no one has solved this area of math" That's not creative prompt. That's a driving prompt to get it to start its engine. You could do that nowadays and while it may spend $1,000 to $100,000 worth of tokens. It will create something humans haven't done before as long as you set it up with all its tool calls/permissions. | | |
| ▲ | awesome_dude 2 hours ago | parent [-] | | Let me know when the Fields medal arrives in the mail. It won't because even though it looks clever to you, people who /do/ understand math and LLMs understand that LLMs /are/ regurgitating Why does your LLM need you to tell it to look in the first place? Why isn't just telling us all the answers to unsolved conjectures known and unknown? Why isn't the LLM just telling us all the answers to all the problems we are facing? Why isn't the LLM telling us, step by step with zero error, how to build the machine that can answer the ultimate question? | | |
| |
| ▲ | charlie90 3 hours ago | parent | prev | next [-] | | I believe when we have AI Agents "living" 24/7, they will become creative machines. They will test ideas out their own ideas experimentally, come across things accidentally, synthesize new ideas. We just haven't let AI run wild yet. But its coming. | | |
| ▲ | awesome_dude 2 hours ago | parent [-] | | So are self-driving cars - as they have been for the last... decade or so AGI has been "just over the horizon" for literal decades now - there have been a number of breakthroughs and AI Winters in the past, and there's no real reason to believe that we've suddenly found the magic potion, when clearly we haven't. AI right now cannot even manage simple /logic/ |
| |
| ▲ | Barbing 4 hours ago | parent | prev [-] | | If that’s a requirement, aren’t LLMs driven by pretraining which was human driven? Who decides at which the last point it’s OK to provide text to the model in order to be able to describe it as creative? (non-rhetorical) |
|
|
