Yep. It's nearly identical to the neural nets we were using in the 90s. Back then even a supercomputer wasn't big enough or fast enough to do what we do today.

I have to wonder though. Is this all a human brain is? A similar thing to an LLM just scaled exponentially larger. I mean a brain is not just neurons with simple connections to each other. The neurons, axons, dendrites, <insert_unexplained_thing>, etc in a brain are all holding and processing information in different ways and doing it nearly 100% in parallel. That's a really big model.

The biological discoveries show how complex a biological brain actually is. Even the tiny brains in a bee or spider are able to solve puzzles and use tools. That's crazy.

▲

ctolsen 3 hours ago | parent | next [-]

No, it’s definitely not what a human brain is. That makes very little sense. The ways we interact with language (and thus conceptual memory) is completely and fundamentally different.

▲

rfv6723 2 hours ago | parent [-]

Is it different though?

If we look beyond written languages which are late inventions of human civilization, oral languages are continuous and build with blocks not words.

Chomskyan school misled the entire field of linguistics for decades by ignoring spoken languages.

	▲	uoaei 2 hours ago \| parent [-]
		It is different, but there may be some universal principles that are relevant more abstractly among both cases. Of particular interest is the empirical notion that statistical models of a certain form will always tend to "average out noise" and "learn meaningful patterns" up to the capacity that those models have for representing said patterns. A parallel notion to this is the hypothesis dubbed "thermodynamic origins of life". The universal principle binding these two seemingly disparate topics is one that seems to underlie any sense of "learning" in physical systems: that semantics of those systems depend on their representational power, and the semantics they do come to represent are the results of adding up many pushes in one "direction" (phase space / state space / etc.) encoding a pattern, and adding up many random noise jiggles will cancel out but give you a first-order sense of variance of those semantic features as expressed by the environment. As this description is so overly abstract, an exercise for the reader is to try to work through an explanation of how, say, a river delta comes to "learn" about its environment by "reacting" to the influences at its borders, and how it "encodes" whatever it is that it learns in the substrate that it inhabits.

▲

redox99 an hour ago | parent | prev | next [-]

In the 90s you didn't have norm layers, residuals, attention, and some more.

So you're missing a lot of the building blocks that make LLMs. It's not a matter of just having the compute.

▲

spacebacon 3 hours ago | parent | prev | next [-]

LLMs are semiotic infrastructure. You won’t find a better analogy. The cognitive frame won’t hold.

▲

bonoboTP 4 hours ago | parent | prev | next [-]

Attention layers were not used in the 90s.

▲

an hour ago | parent | prev | next [-]

[deleted]

▲

otabdeveloper4 3 hours ago | parent | prev | next [-]

> I mean a brain is not just neurons with simple connections to each other.

No, it's not. There are many animals that have extremely complex and even learned behaviour that have literally zero neurons.

Clearly "neurons" is an oversimplification just-so story, not a scientific theory.

	▲	adammarples 41 minutes ago \| parent \| next [-]
		Apparently even single-celled protozoa can show learned trial and error behaviour.
	▲	formerly_proven 2 hours ago \| parent \| prev [-]
		Do you consider fungi animals or do you perhaps mean animals that don't have a brain/CNS?

▲

foxes 5 hours ago | parent | prev [-]

Probably better to not simply reduce it by just saying X is Y then if it has all that extra complexity and capacity.