Remix.run Logo
▲ raincole 10 hours ago

> Now that AGI is coming within reach, we can no longer put off seriously considering what comes next. While far from certain, AI could continue to advance at its current pace - and possibly even accelerate - towards artificial general superintelligence, or ASI

I wonder why they are so uncertain about it. It seems obvious to me that AGI will certainly lead to ASI. Unless human-level intelligence is something like the speed of light?

I don't even believe there is a clear line between AGI and ASI.

▲breuleux 10 hours ago | parent | next [-]

> It seems obvious to me that AGI will certainly lead to ASI. Unless human-level intelligence is something like the speed of light?

It may just be that human-level intelligence is basically "enough" for performing tasks in the natural world, and having more of it doesn't really help except for synthetic, intelligence-shaped tasks like coding. Our perception that intelligence is some kind of unbounded problem-solving system that can do anything if you have enough of it is kinda suspect. There's a good likelihood that AI will wreck the (also intelligence-shaped) social fabric but will be of surprisingly little consequence in the realm of physics and energy.

I think the idea that an intelligence can be "general" is just a conceit borne from the perception of our own success, but mathematically that doesn't actually make any sense. No optimization strategy works on every problem. There must be things that can be intelligently designed that cannot evolve, and things that can evolve but cannot be intelligently designed.

▲skissane 10 hours ago | parent | prev | next [-]

> Unless human-level intelligence is something like the speed of light?

I'm sceptical we will reach fully human-level intelligence simply by scaling current AI approaches, since they are so extremely inefficient compared to human brains, and have other severe limitations such as hard boundaries between training and runtime inference.

Sure, people are working on new architectures that aim to overcome that, but we don't know yet whether those proposals will actually succeed in practice.

▲raincole 8 hours ago | parent [-]

I don't think we'll reach true AGI (by the strictest definition - can do any task that there is at least one human can do) very soon. But if we reached AGI I'm quite certain that it'd reach ASI.

▲xhevahir 10 hours ago | parent | prev [-]

Yeah, the machines will be connected, communicating at lightning speed, and working 24/7. Seems like AGI would make superhuman intelligence a forgone conclusion.