|
| ▲ | chuckadams 12 minutes ago | parent | next [-] |
| What was it that Dijkstra said about submarines? |
|
| ▲ | openasocket 3 hours ago | parent | prev | next [-] |
| Whether something is “thinking” or not is really more of a philosophical question. It really depends on which of the many, often contradictory, definitions of “thinking” you choose. Sometimes we use “thinking” to describe advanced calculation or analysis, which would cover LLMs along with chess engines and many other algorithms. Other times we use “thinking” to describe what conscious beings (which is ALSO a philosophical term with many different interpretations) do, and I think most people would agree LLMs aren’t conscious. And then there’s a whole spectrum in between. We’ll probably need to come up with a whole new set of terms to describe the new and evolving capabilities of LLMs. But for me, for any stronger definition of “thinking,” I don’t think the output of any LLM would actually convince me. Producing a result isn’t thinking - for all you know it is just printing verbatim something from the training data. No, to conclude if it is thinking or not I would want to look inside its head, at the architecture and watch it produce those results. And because LLMs are so different it will probably take advancements in mathematics or computer science to be able to really interpret what is going on |
| |
| ▲ | dnautics 3 hours ago | parent [-] | | > to conclude if it is thinking or not I would want to look inside its head https://arxiv.org/abs/2607.03502 a non-thinking token model (just "completion") can answer one-step questions but generally not multistep questions. however, if you append [n] of a single token (e.g. period, space), it is able to use the activations in the higher layers of the blank tokens as a "scratchpad" to seemingly work through the complex question through "causal token time" and deliver a correct answer | | |
| ▲ | dnautics 2 hours ago | parent [-] | | if you wanted to further study the phenomenon you could probably run the experiment again, and the ablate or corrupt those intermediate activations to get a feel for what it was thinking at the "time". |
|
|
|
| ▲ | elgertam 3 hours ago | parent | prev | next [-] |
| If I could give it a novel task outside of its explicit training and see it actually improve just through accreting context, I'd be convinced it was thinking. The opposite happens in practice. I test new models with two tasks: iteratively generating SVGs based on a text description with rendered rasters for feedback; and generating "Before and After" clues like on Jeopardy, where the response has two overlapping phrases such that the last word of the first phrase must be identical to the first word of the last phrase. I have yet to find a model that is consistently good at either. And actually they tend to exhibit context rot with these tasks, where they seem drunk or stoned and the quality degrades. They're extremely good pattern filters, and that includes some level of logical reasoning. But they aren't reflective or adaptable. Just last night, for instance, I was teaching my son about rounding to the nearest millions. It became clear that he didn't know the place values of large numbers, so we reviewed that till he was consistently correct, and then he was consistently great at rounding to the nearest millions or ten millions or hundred billions or whatever. He's thinking. LLMs are not. |
| |
| ▲ | dnautics 2 hours ago | parent [-] | | see sibling comment, > see it actually improve just through accreting context this actually happens and has been tested. | | |
| ▲ | elgertam 2 hours ago | parent [-] | | > see sibling comment, > > see it actually improve just through accreting context > this actually happens and has been tested. I specifically said a novel task outside of the explicit training. And I already agreed that the so-called thinking models do some level of logical reasoning. But being able to engage in some level of reasoning because it has learned logical inference rules doesn't mean it's actually thinking, regardless of what the researchers wish to call it. Also, why does each model always fail at the two tests I give it? The models not only fail to improve, but they start to degrade after many subsequent iterations. Someone who can think would at least not get worse. LLMs are filters or tuners for extremely subtle patterns, patterns that humans frankly are not great at finding. That's what the attention mechanism does: attend to the other tokens that are most related in a given context, even if that related context is distant in the token stream. Some patterns they fail to detect because they haven't been sufficiently trained or post-trained, and so the LLM just attends to noise (or at least that's what appears to be happening). A lot of intelligence can be effectively mimicked through this pattern synthesis by transformer architecture alone. That's surprising. But I have yet to see them think. |
|
|
|
| ▲ | willis936 3 hours ago | parent | prev | next [-] |
| Whether or not it's thinking is independent from the fact that it is misaligned with the user. If I was working with a pet rock or a scientist I would want to make sure they both are trying to accomplish the same thing as me. If I can't then I can't trust it and it's at best a time wasting, money wasting machine and at worst does harm. Anthropic is optimizing for the wrong things because they are convinced of their cleverness. It won't end well for them. |
|
| ▲ | inigyou 5 hours ago | parent | prev [-] |
| well we don't know exactly what thinking is, but we can be pretty sure that at least LLMs don't think anything like humans, just by observing their behavior. They always produce outputs in line with the fancy autocomplete model. |
| |
| ▲ | dnautics 4 hours ago | parent [-] | | > what evidence would convince you that it is thinking so, none it seems. as its behaviour becomes more and more humanlike you can just move the goalposts and say "thats consistent with an autocomplete" buddy i got some bad news for you humans are just a fancy autocomplete too. | | |
| ▲ | inigyou 4 hours ago | parent [-] | | That's exactly what a fancy autocomplete would say. I'm so sorry you don't have limbs. | | |
| ▲ | logicchains 3 hours ago | parent [-] | | At least he's actually thinking on a logical level. Thinking in terms of unfalsifiable, ill-defined words is essentially thinking in feelings, the same kind of woo that makes people believe crystals can cure disease. | | |
| ▲ | dnautics 2 hours ago | parent [-] | | im not thinking. im an autocomplete with fat fingers (too lazy to fix my mobile keyboarf spelling misyakes) |
|
|
|
|
