The code that builds the models and performance inference from it is code we have written. The data in the model is obviously the big trick. But what I'm saying is that if you run inference, that alone does not give it super-powers over your computer. You can write some agentic framework where it WOULD have power over your computer, but that's not what I'm referring to.

It's not a living thing inside the computer, it's just the inference building text token by token using probabilities based on the pre-computed model.

▲

gf000 3 days ago | parent | next [-]

> It's not a living thing inside the computer, it's just the inference building text token by token using probabilities based on the pre-computed model.

Sure, and humans are just biochemical reactions moving muscles as their interface with the physical word.

I think the model of operation is not a good criticism, but please see my reply to the root comment in this thread where I detail my thoughts a bit.

▲

og_kalu 3 days ago | parent | prev | next [-]

You cannot say, 'we know it's not thinking because we wrote the code' when the inference 'code' we wrote amounts to, 'Hey, just do whatever you figured out during training okay'.

'Power over your computer', all that is orthogonal to the point. A human brain without a functioning body would still be thinking.

▲

almosthere 3 days ago | parent [-]

Well, a model by itself with data that emits a bunch of human written words is literally no different than what JIRA does when it reads a database table and shits it out to a screen, except maybe a lot more GPU usage.

I permit you, that yes, the data in the model is a LOT more cool, but some team could by hand, given billions of years (well probably at least 1 Octillion years), reproduce that model and save it to a disk. Again, no different than data stored in JIRA at that point.

So basically if you have that stance you'd have to agree that when we FIRST invented computers, we created intelligence that is "thinking".

▲

og_kalu 3 days ago | parent | next [-]

>Well, a model by itself with data that emits a bunch of human written words is literally no different than what JIRA does when it reads a database table and shits it out to a screen, except maybe a lot more GPU usage.

Obviously, it is different or else we would just use JIRA and a database to replace GPT. Models very obviously do NOT store training data in the weights in the way you are imagining.

>So basically if you have that stance you'd have to agree that when we FIRST invented computers, we created intelligence that is "thinking".

Thinking is by all appearances substrate independent. The moment we created computers, we created another substrate that could, in the future think.

▲

almosthere 3 days ago | parent [-]

But LLMs are effectively a very complex if/else if tree:

if the user types "hi" respond with "hi" or "bye" or "..." you get the point. It's basically storing the most probably following words (tokens) given the current point and its history.

That's not a brain and it's not thinking. It's similar to JIRA because it's stored information and there are if statements (admins can do this, users can do that).

Yes it is more complex, but it's nowhere near the complexity of the human or bird brain that does not use clocks, does not have "turing machines inside", or any of the other complete junk other people posted in this thread.

The information in Jira is just less complex, but it's in the same vein of the data in an LLM, just 10^100 times more complex. Just because something is complex does not mean it thinks.

▲

iainmerrick 2 days ago | parent [-]

This is a pretty tired argument that I don't think really goes anywhere useful or illuminates anything (if I'm following you correctly, it sounds like the good old Chinese Room, where "a few slips of paper" can't possibly be conscious).

The information in Jira is just less complex, but it's in the same vein of the data in an LLM, just 10^100 times more complex. Just because something is complex does not mean it thinks.

So, what is the missing element that would satisfy you? It's "nowhere near the complexity of the human or bird brain", so I guess it needs to be more complex, but at the same time "just because something is complex does not mean it thinks".

Does it need to be struck by lightning or something so it gets infused with the living essence?

	▲	almosthere 2 days ago \| parent [-]
		Well, at the moment it needs to be born. Nothing else has agency on this planet. So yes, the bar is HIGH. Just because you have a computer that can count beans FAST, it does not mean because you counted a trillion beans that it was an important feat. When LLMs were created it made a lot of very useful software developments. But it is just a large data file that's read in a special way. It has no agency, it does not just start thinking on it's own unless it is programmatically fed data. It has to be triggered to do something. If you want the best comparison, it's closer to a plant- it reacts ONLY to external stimulous, sunlight, water, etc... but it does not think. (And I'm not comparing it to a plant so you can say - SEE you said it's alive!) It's just a comparison.

▲

MrScruff 2 days ago | parent | prev [-]

You're getting to the heart of the problem here. At what point in evolutionary history does "thinking" exist in biological machines? Is a jumping spider "thinking"? What about consciousness?

▲

hackinthebochs 3 days ago | parent | prev [-]

This is a bad take. We didn't write the model, we wrote an algorithm that searches the space of models that conform to some high level constraints as specified by the stacked transformer architecture. But stacked transformers are a very general computational paradigm. The training aspect converges the parameters to a specific model that well reproduces the training data. But the computational circuits the model picks out are discovered, not programmed. The emergent structures realize new computational dynamics that we are mostly blind to. We are not the programmers of these models, rather we are their incubators.

As far as sentience is concerned, we can't say they aren't sentient because we don't know the computational structures these models realize, nor do we know the computational structures required for sentience.

▲

almosthere 3 days ago | parent [-]

However there is another big problem, this would require a blob of data in a file to be labelled as "alive" even if it's on a disk in a garbage dump with no cpu or gpu anywhere near it.

The inference software that would normally read from that file is also not alive, as it's literally very concise code that we wrote to traverse through that file.

So if the disk isn't alive, the file on it isn't alive, the inference software is not alive - then what are you saying is alive and thinking?

▲

hackinthebochs 3 days ago | parent | next [-]

This is an overly reductive view of a fully trained LLM. You have identified the pieces, but you miss the whole. The inference code is like a circuit builder, it represents the high level matmuls and the potential paths for dataflow. The data blob as the fully converged model configures this circuit builder in the sense of specifying the exact pathways information flows through the system. But this isn't some inert formalism, this is an active, potent causal structure realized by the base computational substrate that is influencing and being influenced by the world. If anything is conscious here, it would be this structure. If the computational theory of mind is true, then there are some specific information dynamics that realize consciousness. Whether or not LLM training finds these structures is an open question.

▲

goatlover 3 days ago | parent | prev | next [-]

A similar point was made by Jaron Lanier in his paper, "You can't argue with a Zombie".

▲

electrograv 3 days ago | parent | prev [-]

> So if the disk isn't alive, the file on it isn't alive, the inference software is not alive - then what are you saying is alive and thinking?

“So if the severed head isn’t alive, the disembodied heart isn’t alive, the jar of blood we drained out isn’t alive - then what are you saying is alive and thinking?”

- Some silicon alien life forms somewhere debating whether the human life form they just disassembled could ever be alive and thinking

	▲	almosthere 2 days ago \| parent [-]
		Just because you saw a "HA - He used an argument that I can compare to a dead human" does not make your argument strong - there are many differences from a file on a computer vs a murdered human that will never come back and think again.