So AI makes it cheaper to remix anything already-seen, or anything with a stable pattern, if you’re willing to throw enough resources at it.

AI makes it cheap (eventually almost free) to traverse the already-discovered and reach the edge of uncharted territory. If we think of a sphere, where we start at the center, and the surface is the edge of uncharted territory, then AI lets you move instantly to the surface.

If anything solved becomes cheap to re-instantiate, does R&D reach a point where it can’t ever pay off? Why would one pay for the long-researched thing when they can get it for free tomorrow? There will be some value in having it today, just like having knowledge about a stock today is more valuable than the same knowledge learned tomorrow. But does value itself go away for anything digital, and only remain for anything non-copyable?

The volume of a sphere grows faster than the surface area. But if traversing the interior is instant and frictionless, what does that imply?

▲

ukuina 20 minutes ago | parent | next [-]

Single-idea implementations ("one-trick ponies") will die off, and composites that are harder to disassemble will be worth more.

▲

tornikeo an hour ago | parent | prev | next [-]

> The volume of a sphere grows faster than the surface area. But if traversing the interior is instant and frictionless, what does that imply?

It's nearly frictionless, not frictionless because someone has to use the output (or at least verify it works). Also, why do you think the "shape" of the knowledge is spherical? I don't assume to know the shape but whatever it is, it has to be a fractal-like, branching, repeating pattern.

▲

ramraj07 an hour ago | parent | prev [-]

The fundamental idea that modern LLMs can only ever remix, even if its technically true (doubt), in my opinion only says to me that all knowledge is only ever a remix, perhaps even mathematically so. Anyone who still keeps implying these are statistical parrots or whatever is just going to regret these decisions in the future.

▲

pseudosavant 35 minutes ago | parent | next [-]

But all of my great ideas are purely from my own original inspiration, and not learning or pattern matching. Nothing derivative or remixed. /sarcasm

▲

mrbungie an hour ago | parent | prev | next [-]

> Anyone who still keeps implying these are statistical parrots or whatever is just going to regret these decisions in the future.

You know this is a false dichotomy right? You can treat and consider LLMs statistical parrots and at the same time take advantage of them.

▲

heavyset_go an hour ago | parent | prev [-]

Yeah, Yann LeCun is just some luddite lol

▲

NitpickLawyer 8 minutes ago | parent [-]

I don't think he's a luddite at all. He's brilliant in what he does, but he can also be wrong in his predictions (as are all humans from time to time). He did have 3 main predictions in ~23-24 that turned out to be wrong in hindsight. Debatable why they were wrong, but yeah.

In a stage interview (a bit after the "sparks of agi in gpt4" paper came out) he made 3 statemets:

a) llms can't do math. They can trick us with poems and subjective prose, but at objective math they fail.

b) they can't plan

c) by the nature of their autoregressive architecture, errors compound. so a wrong token will make their output irreversibly wrong, and spiral out of control.

I think we can safely say that all of these turned out to be wrong. It's very possible that he meant something more abstract, and technical at its core, but in the real life all of these things were overcome. So, not a luddite, but also not a seer.

	▲	gjadi 2 minutes ago \| parent [-]
		Have this shortcomings of llms been addressed by better models or by better integration with other tools? Like, are they better at coding because the models are truly better or because the agentic loops are better designed?