A 5090 has a peak theoretical limit of GenAI 3356TOPS. So we're "already" an order of magnitude greater than what was considered enough for AGI. One question is, "What happened here?" Was the original estimate wrong? Have we notfound the "right" algorithm yet? Something else?

▲

lukeschlather 5 days ago | parent | next [-]

"We haven't found the 'right' algorithm yet." seems like the obvious answer, but the numbers in the paper all make sense and I'm interested in some more exotic explanations why it could actually be some orders of magnitude more than a 5090.

Although that's not looking at memory, and I am also interested in some explanation of what... a 5090 has 32GB which, a human brain has more like a petabyte of memory assuming 1 byte/synapse. Which is to say 1 million GB in which case even a large cluster of H100s has an absurd amount of TOPS but nowhere near enough high-speed memory.

▲

nvch 5 days ago | parent | prev | next [-]

We are constantly learning (updating the network) in addition to inference. Quite possibly that our brains allocate more resources to learning than to inference.

Perhaps AI companies don’t know how to run continuous learning on their models:

* it’s unrealistic to do it for one big model because it will instantly start shifting in an unknown direction

* they can’t make millions of clones of their model, run them separately and set them free like it happens with humans

	▲	Mars008 4 days ago \| parent [-]
		> possibly that our brains allocate more resources to learning than to inference It's likely in brain inference is learning. If you want a technical analog it's like a conversation in LLMs. Previous tokens do affect the currently generated. I.e. it's inference time learning, well known and widely used.

▲

SoftTalker 5 days ago | parent | prev [-]

Nature needed 3.5 billion years to work it out, and we're going to solve it in a few decades?

▲

kelseyfrog 5 days ago | parent | next [-]

It depends on where we draw the starting line. We're already at parity with 3.5BYA to 541Mya because no neurons existed in that duration. Only more recently, in the Cambrian, do we have evidence that voltage gated potassium signaling evolved[1].

That changes the calculus likely very little, but it feels more accurate.

1. https://www.cell.com/current-biology/pdf/S0960-9822(16)30489...

▲

mathgeek 5 days ago | parent | prev | next [-]

I know it’s a silly question to begin with, but if you analyze it seriously, you’d want to at most compare human intelligence->superintelligence with the 20 million years between the first homidinae and homo (and even that is probably too large for some folks to compare with).

One could even argue you should only compare it back to the discovery of writing or similar.

▲

Jyaif 5 days ago | parent | prev [-]

That's not an argument. Nature never worked out going into space, yet we solved it in a few decades.

▲

jll29 5 days ago | parent | next [-]

Yes but that's "in a few decades" ON TOP of millions of years.

If I had to give an estimate, I would consider less the time taken to date, but the current state of our knowledge of how the brain works, and how it has grown in the last decades. There is almost nothing that we know so little about as the human brain, how thoughts are represented, modern imaging techniques notwithstanding.

	▲	exe34 5 days ago \| parent [-]
		> Yes but that's "in a few decades" ON TOP of millions of years. If that's the bar, then anything else can fit in "a few decades", since that also rests "ON TOP of millions of years".

▲

SoftTalker 5 days ago | parent | prev | next [-]

It worked out flying though, millions of years before we did and we still don't do it as well. We can't even do walking as well as nature did.

	▲	baq 5 days ago \| parent \| next [-]
		Walking is easy compared to elbows, fingers and thumbs. It’s just falling over in a controlled fashion. I hear at least one company in Boston figured it out. Anyway, humanoid robots should be big in the next 10-20 years. The compute, the batteries, the algorithms are all coming together.
	▲	derektank 5 days ago \| parent \| prev [-]
		We do flying better. If you adjust for our body weight, a modern airliner uses less energy per traveller mile than your average migratory bird. And the airliner goes much faster.

▲

gnz11 5 days ago | parent | prev [-]

One could argue nature solved it by evolving homo sapiens.