Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ TalkingCodeMonk 5 hours ago

The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever.

No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can not easily be explained by "correlation engine", including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection.

▲PaulHoule 5 hours ago | parent | next [-]

Some of it the effect of tells. “It’s not X, it’s Y” is not a bad pattern but it was baked into the instruction following training set just like the other patterns. I catch myself about to use it and use something else because I want to look human. I have, a few times, tried to use AI to write something that I was struggling to find the words and I just didn’t like how it didn’t seem like my voice. If there was just one person doing it would be OK but when it is 100s of blog posts submitted to HN a day it is like wearing a “I’m an NPC” t-shirt.

▲sa-code an hour ago | parent [-]

Someone shared with me this system prompt that at least makes assistant outputs usable

    For information retrieval tasks, I want you to provide links to sources and use exact quotes as much as possible. When using a source, consider if it is primary or secondary information. If secondary sources are found, search again for primary sources. Sources and quotes, if applicable, should be mentioned in the answer first before the rest of the response with links.
▲pyridines 5 hours ago | parent | prev | next [-]

I just can't accept that it possesses no intelligence. It is not equivalent to human intelligence, obviously, but how can a system without some semblance of rational thinking solve open math problems? Even composing earlier human work into something novel requires intelligence and understanding on some level.

▲wongarsu 4 hours ago | parent | next [-]

We couldn't agree on what intelligence means before ChatGPT happened. Now, agreement on the term seems even further away

If performing well on an IQ test or performing at a high level on knowledge work is intelligence to you, these models are intelligent. If intelligence requires sentience for you, then ... well, I don't think we really agree what that is either, never mind how to measure it. But LLMs certainly don't have it right now

But the consistent trend of the last couple decades (arguably since Turing's time) seems to be that any time a computer reaches our definition of intelligence we decide that that was a flawed definition

▲bluetomcat 4 hours ago | parent | prev | next [-]

It has no semantic depth. The sentences and the paragraphs are a statistically viable derivation of existing human text, but once you try to grasp the whole thing with its temporal and spatial dimensions, you are left with a blurry mess that rots your brain. It's a polished, inoffensive and shallow interpretation as written by an opinionated reputation-seeking user of Quora, circa 2019. Assertive, bold, without typos, clean-cut and bulleted, but without an interesting semantic core.

▲bayindirh 5 hours ago | parent | prev | next [-]

It's just filled to the brim with relations between things. It's good at searching a very large meaning space and create correlations. What it does is to cover great distances and find related things in that large space which needs a long time and large corpus of knowledge to find the connection.

This is not intelligence. It's just a good correlation engine with a very big albeit lossy database of things.

▲rnd33 4 hours ago | parent | next [-]

Intelligence is compression, compression requires subtraction, and for some reason LLMs are not good at subtracting. To create a coherent model you kinda have to subtract correlations until only the essential parts are still there.

What I don't understand is why LLMs haven't been able to do this yet, if it's the harness or some orchestration layer above the LLM that is needed. Because fundamentally if you can identify correlations then it's just another small step to prioritize and remove lower value or irrelevant correlations.

I wonder if what's needed is to introduce subtraction tokens in some sense, and in post-training reward the model on that.

▲tyromaniac a minute ago | parent | next [-]

I've heard that expression before, but I don't think it can be presented and stated so matter of factly. Where does that put bzip?

▲fluoridation an hour ago | parent | prev | next [-]

Intelligence is compression? What do you mean? Intuitively that doesn't seem right.

>What I don't understand is why LLMs haven't been able to do this yet

LLMs are just trained on what humans have said. Why is it surprising that it's still not possible to reconstruct the intelligence that wrote all that by working backwards? Think of your own work experience. When you look at a piece of code, say, are you always able to discern why the person did what they did, just from the code, with no additional context?

▲jrmg 2 hours ago | parent | prev [-]

Intelligence is compression

That’s a controversial statement.

▲pyridines 5 hours ago | parent | prev [-]

The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, for something that is just one big matrix multiplication. Drawing connections between things in meaning-space is surely a facet of intelligence.

▲bayindirh 4 hours ago | parent | next [-]

It’s not intelligence if you are the one who gives the correlations to the model in the pre-training. It’s Word2Vec, applied. Model doesn’t learn anything. You embed these correlations and build it from there. It just searches the space.

As my AI professor said in the first lecture: “All AI is advanced search”.

▲pyridines an hour ago | parent | next [-]

Okay, I guess you're right that its ability to do this is just correlational, which doesn't imply it has any understanding. However, you have to conclude that some tasks which we used to believe required intelligence don't actually require any, which is disconcerting.

▲bayindirh an hour ago | parent [-]

No, what I would say is the tasks which are handled in a passable manner by LLMs can be mathematically modeled with some reasonable accuracy.

Many things are predicted by models in our planet. From weather to production and material science. Building the model needs intelligence, running the model does not.

The person who came up with the formulae for CFD was intelligent. The computer running the model is not. Same for LLMs, chess engines, engine ECUs and financial prediction systems.

Again, for the example’s sake; the person who came up with an algorithm is intelligent. The model mixing its training data to emit something similar is not.

▲pyridines 25 minutes ago | parent | next [-]

I get what you're saying. The thing itself is just math. I'll just say it depends on how you define intelligence. If at some point we're be able to simulate a human brain with 100% accuracy, I would say that it is intelligent, it sounds like you would not. (I don't mean to imply consciousness or personhood or anything else by "intelligent".)

▲hombre_fatal 25 minutes ago | parent | prev [-]

This starts to feel like you're defining the word intelligence out of any meaning and out of any way we apply that word.

So when LLMs can do all human knowledge work, and do it better than humans, we'll be in the mines listening to you go on about how it's actually just autocomplete or just math, a distinction that apparently means nothing.

▲ 2 hours ago | parent | prev [-]
[deleted]
▲ 4 hours ago | parent | prev [-]
[deleted]
▲duped an hour ago | parent | prev | next [-]

I don't think statistically driven prediction implies reasoning or intelligence.

▲malfist 35 minutes ago | parent [-]

Its a mirror to human intelligence. Regurgitating phrasing to match what someone who can reason put together, but it isn't any more intelligent than the reflection of you in the mirror is.

▲figers 5 hours ago | parent | prev | next [-]

watch this and see if you think it has intelligence by the end

https://www.youtube.com/watch?v=kYUicaho5k8

▲ragequittah 4 hours ago | parent | next [-]

I wonder if you went back before we had any idea how the brain worked and talked to the smartest people about how neurons work (without giving away that it's a human brain) then asked them all "would such a system be intelligent?" how many would say yes.

The main problem I have with people stating it's not intelligent or conscious is I don't think we even have a good definition of either word that satisfies everyone. Philosophers have been trying (and failing) to elegantly define these things forever and everyone out here proclaiming they've got the definitive answer and this specific thing they're seeing doesn't fit under it.

▲pyridines 5 hours ago | parent | prev | next [-]

This looks interesting, but would you mind saying a sentence or two about why before I commit to an hour-long video? It looks like it shows how they work internally, which is sort of a non sequitur. Brains also work mechanistically. I'm claiming that any system which is able to do what AIs do must necessarily have some sort of intelligence.

▲figers 4 hours ago | parent [-]

fair reply to an hour video, Scott is just so good to hear his talk is better than I can explain it...

go to 24 minutes and 07 seconds.

it's statistically determining what the next word should be based on all the text it's been trained on. It's not intelligence and he shows what probability it puts on each word that it chooses, but also shows a lot of the other words it was thinking of using. In a later part he shows how it uses words that are not the highest probability (and you question why did it go this route, it's not more correct), but the user never sees this, they see what they think is the correct answer always...

he also shows how context you feed it has a lot to do with what it returns... to the point he can get it to return the capital of France is Marseille, just by typing Marseille a bunch of times before the question. Human intelligence doesn't get confused like that.

And it's not a "hallucination", it's just probability of the next token prediction based on the information it's been trained on and fed, it's not intelligence.

▲rnd33 4 hours ago | parent [-]

Isn't this a case of missing the trees for the forest though? The human brain is not an LLM, and an LLM is not intelligent in the same way as a human brain.

However, an LLM is a prediction machine, prediction IS at the very least one (or the most fundamental) element of intelligence. The brain most surely contains at least some kind of simulacrum of a prediction machine. How that prediction machine is used or wrapped is another matter.

If I said to you: "Blue blue blue, the color of my car is red", would you have absolute confidence in your prediction that my car is red? Or would the way I phrased that sentence make you slightly uncertain, and wonder if there's some miscommunication going on here?

▲figers 3 hours ago | parent [-]

LLMs are awesome awesome tech!

A lot of people seem to think it's human level intelligence.

▲bayindirh 5 hours ago | parent | prev [-]

I also like this: https://laurentiugabriel.github.io/token-town/

It shows internals of an LLM nicely, simplified manner.

▲ThrowawayR2 4 hours ago | parent | prev [-]

LLMs are pattern prediction systems with a large training data set. It is not surprising that they can predict patterns, particularly for a well structured field like mathematics that is also amenable to automated proof checking to help steer it.

▲fallingbananna 30 minutes ago | parent | prev | next [-]

Are we sure there is some objective, technical definition of what is intelligence and what is not?

Isn't it rather a subjective philosophical concept? What if human intelligence is also a statistical model, trained by evolution to make decisions that lead to offspring?

The one major difference I see between AI and people is the ability to learn and memorize. All memory/learning solutions that current AI architectures offer just feel like workarounds and simply don't work anywhere near as a person learning something new and remembering it.

▲tempodox 5 hours ago | parent | prev | next [-]

Thank you for helping me keep my sanity.

▲GPerson 4 hours ago | parent | prev [-]

I mean this in the kindest way possible, but you are wrong that the math solutions are that easily dismissed. And there are many more than are publicized. A specific math problem I wanted solved for 3 years did not get solved by any model until fable and, and I tried it on every model and know the literature surrounding it well.