> It cant "invent new things"

I've heard that before but it just doesn't make sense in context of what I've seen llms do. If I ask an llm to write a poem about magnetic resonance and vampire rabbits it can do that it created a new thing. I can ask it to build a website for managing rabbit breeding that's also a new thing.

Another way of looking at it is that human beings, just like llms, can produce output based on their inputs. Most literature is inspired by other literature. Most music is inspired by other music. Most software is inspired by other software.

So I think we need to work on defining " new things" before we can definitely exclude them from llm's capabilities

I'm not sure how you'd land on an LLM being unable to invent anything new. You'd have to both understand exactly what LLMs are and aren't capable of today and what allows human creativity. I don't think there are good answers to either.

It seems unlikely that recursively predicting the next word would lead to creativity or invention, but it doesn't seem impossible. Similarly, it seems unlikely that human thought works in a similar prediction loop, but it doesn't seem impossible.

> then it turned out it was really just listening in on a math professors supposed-to-be-private conversations

No, it didn't turn out to be that. Someone made a claim, which is silly for many reasons. There's no serious support for this happening.

I just don’t know enough to agree with you, but I would argue, probably poorly, that an AI or collection of agents could “invent things” simply by virtue of trying essentially everything in a reasonable bound and stumbling into a solution.

Call that brute force perhaps, but I would consider it technically inventing something on the merit that it would at least be an abstraction above naively throwing everything against a wall to only throwing things that would most likely be sticky.

You're overconfident in what you're saying. Nobody really knows (yet) if LLMs can or cannot "think" or can or cannot "invent" new things. I'm inclined to think the opposite from you but smart people actually try not to hold the absolut stances and present them as facts when they're in fact not. What I have seen so far throughout a daily use on complicated things suggests that humanity developed a new form of an intelligence.

You’re assuming the “listening in” aspect, which as far as I know is not proven (even in the less loaded “the model was trained on sessions including the ones in question” form of the claim.

Plausible? Absolutely. Did OpenAI behave badly in other ways regarding this issue? Yes. Does it help to assume unproven facts and then accuse people of reaching emotional decisions? Nope.

i think the proper understanding of LLM based AI is "contexting"; we describe the world we want them to fill, we build/direct the outside context for them to understand, then "they" take off from there.

If we do a bad job building context, they do a horrible job contexting. People who have trouble working with AI have the same problem people have in general: if they can't figure out the context of the direction, then they make random decisions of doing anything. On the flip side, if you can build the proper context around a sufficiently powerful LLM, they can derive the context via the contexting they're good at.

This is why building documents, tests, and code all in some intent pattern via prompting allows them to do a significant amount of work a normal person would have a great effort t

[dead]

[flagged]

Given a budget and the right amount of access, I see no reason why an LLM/agent couldn't right now order parts from a "supplier", instruct the human meat machine to insert the device onto a connected test bed, and iterate. This is essentially what Claude Fable can do currently with any connected device and tools it had access to.

The repo claims they tested with an FPGA. To be fair it is not the same as an ASIC, and we have no idea what errata abound, but it actually did make a thing that spat out tokens.

Maybe this is the level of simulation we need: https://qntm.org/responsibilit

That’s fair, I have no idea if this simulator is actually accurate but I appreciate the response.

> They are limited to the sensor's margin of error, range and trusting it's working correctly

So are we and our sensors can be pretty vague in comparison, I can only imagine human error correction is pretty next level.

> They can't confirm ground truth about physical reality.

That's true. Of course we can hook an llm up to Motors and sensors. That's a robot. Or a self-driving car. So would you say that those devices can confirm the ground truth about physical reality, and therefore are capable of creativity?

What is ground truth? If I see a digital painting by a human is that ground truth?