What's "thinking"? What's "agency"? What's "human-like agency"?

If "agency" is making decisions and performing corresponding actions in the real world, then LLMs most definitely LOOK LIKE they're making decisions (what's the next token? which tool to use? what's to say, in general? what idea to convey?) and performing actions (tool use). Can we tell whether they are ACTUALLY making decisions? Well, are the people around me "actually" making decisions? Or are they simply pushed around by circumstances and external forces?

Am I actually making decisions? Did I like DECIDE to write this comment? Maybe? I have no clue...

▲

operatingthetan an hour ago | parent [-]

I think you're mildly obfuscating the issues at hand by diving too deeply into philosophical questions.

It's quite simple, the agency that the LLM appears to have is actually your own. Without a prompt an LLM does nothing. It has no thoughts between prompts about you or your problems.

▲

ForceBru 39 minutes ago | parent | next [-]

Yes, I'm diving a bit too deeply because I don't really know what "thinking" is and therefore I don't understand how we can so confidently say that LLMs don't think, even though they definitely LOOK like they're thinking. They even have a "Thinking" section in their responses! If I say that a rock doesn't think, it's pretty convincing: does a rock look like it's thinking? No — it doesn't even do anything! But an LLM does look like it's thinking, at least while generating a response. When it's "offline" it's just a bunch of "dead" bytes, sure.

So when it's not active, not responding to a prompt, it's of course not thinking. I'm pretty sure nobody actually questions this. Is your computer "thinking" when it's powered off? Can a piece of metal think? Probably not. So there are no thoughts between prompts, this seems obvious.

Thus, this is a question of "discrete time vs continuous time". LLMs "live" from prompt to prompt. Humans are alive continuously. In some sense, we're prompted by a lot of things all the time. As I'm writing this, I'm seeing stuff, I'm hearing stuff, I can feel various parts of my body, I'm thinking about my problems, my goals, other people's problems and goals, etc. When I'm in a sensory deprivation tank, my brain keeps "entertaining" me by "self-prompting", like a recurrent neural network (I guess it literally is a massive RNN).

So it seems like your definition of "thinking" hinges upon the LLMs being discrete-time and single-threaded (can't think about multiple things in parallel).

IMO a more interesting question is whether an LLM is thinking WHILE IT'S GENERATING A RESPONSE, while it's "alive".

	▲	operatingthetan 6 minutes ago \| parent \| next [-]
		I want to say I really appreciate that you are putting a lot of thought into this, you certainly have interesting concepts here. However I think it seems a bit far off from the discussion I'm trying to have, and I do not have the bandwidth to fully understand and charitably respond to your points.
	▲	Shitty-kitty 6 minutes ago \| parent \| prev [-]
		We don't know what thinking is but pattern matching is definitely a big part of it. That's why people see Jesus on a piece of burnt toast.

▲

aspenmartin an hour ago | parent | prev [-]

You are implying definitions that don't seem to be mainstream; thinking is internally manipulating information to reason, infer, plan, solve problems, and form judgments or beliefs. Also -- "Without a prompt an LLM does nothing. It has no thoughts between prompts about you or your problems." it sounds like you paint this like it's something fundamental? It isn't. Nothing is stopping you from streaming information to an LLM and letting it process this information, this is precisely what people are trying to build.

	▲	operatingthetan an hour ago \| parent [-]
		The machines have no driving force to act in the world. That is fundamental for humans. Twice in your comment you suggest things that you think that I believe, please do not do this.