Remix.run Logo
▲ hn_submit 41 minutes ago

After AGI comes Superintelligence or ASI. I thought that was already well established as a term. I believe AGI and ASI are very much related since if you can achieve AGI simply increasing compute resources will give you ASI. OTOH since we're very far away from AGI I'm not sure if it makes sense to already contemplate its effects. We aren't pondering what the effects of nuclear fusion and limitless power or quantum computing would be, are we?

AGI means a cognitive capability comparable to human beings in all aspects to me. Thus that includes sample efficiency, concept extraction, continuous learning and power efficiency. Although the latter can be skipped if the gains are high enough.

I'm not convinced that scaling training data will lead to AGI. A paradigm shift obviously could lead to AGI because it's unknowable what that might be and therefore the least predicable.

Recursive Self Improvement has often been cited as a possible venue, but no one currently knows how to do that with LLMs and Transformers.

▲throwaway384034 36 minutes ago | parent | next [-]

Intelligence is grown and it will continue to grow. Thinking intelligence can be designed is walking into the same dead end many people before you already have.

▲hn_submit 23 minutes ago | parent [-]

I still remember this guy in the 1980's who had the idea to make a database with all known "facts" in it which he believed could be used by computers to allow them to reason. He painstakingly built this database by hand and worked on it for decades (was it called "Cog" or something?)

When I read about LLMs and massive amounts of training data being fed to neural networks around 2020 I realized that time had finally caught up with him.

▲throwaway384034 5 minutes ago | parent | next [-]

Most training is now post-training using RL not human knowledge: https://pbs.twimg.com/media/HUSAcpFbsAAevEh?format=png&name=...

Having some built in knowledge of the world through pre-training is only part of the bootstrapping process.

▲mysterydip 19 minutes ago | parent | prev [-]

Not sure if it’s the same thing you’re thinking of, but Cyc and OpenCyc are like that. I used them in an AI class 20 years ago: https://standards.clarin.eu/sis/views/view-spec.xq?id=SpecOp...

▲hn_submit 16 minutes ago | parent [-]

Yes, I believe that's it.

▲MSFT_Edging 10 minutes ago | parent | prev | next [-]

> We aren't pondering what the effects of nuclear fusion and limitless power or quantum computing would be, are we?

What? All the time!

We've been seeing discussions on post-quantum encryption for a decade or two. The effects of nuclear fusion have been in the background of countless scifi stories, the idea of an extreme amount of abundant energy unlocks 100s of "that can't happen due to energy density" dead-end what-ifs.

Additionally, we're pondering the effects of AGI/SGI because a handful of extremely rich people have been telling us we'll all be out of all our jobs in x years, for almost 6 years now.

They've literally been sitting in press releases telling us "Enjoy that comfort because you won't be able to feed your family when you're outsourced. Your work and career will be worthless, and we're leading the way towards returning you to peasant life!"

Regardless of how out-of-reach AGI is right now, people will continue to work on it unless we end up in some Butlerian jihad situation. Just like how fusion is still out of reach, but far closer. They've got fusion reactors running for over a minute now! 10 years ago they could barely get the metaphorical engine to turn over.

Why wouldn't we ponder the effects? Do you walk through doors without consideration of what's on the other side?

▲exe34 11 minutes ago | parent | prev [-]

> AGI means a cognitive capability comparable to human beings in all aspects to me

So a medical doctor and a structural engineer can never be considered general intelligence separately?

What we have is spiky ASI. It's really good at a huge number of things that nobody can be good at simultaneously. It also sucks at some stuff that's really obvious and easy for us.

▲win311fwg a few seconds ago | parent [-]

> So a medical doctor and a structural engineer can never be considered general intelligence separately?

Why not? Cognitive capability usually refers to one's capacity to think, reason, learn, and process information. Presumably, on average, medical doctors and structural engineers are pretty comparable on that front.

Cognitive capacity is not dependent on your posteriori knowledge.