You consider AI in isolation but never consider how humans might be incentivized to "help them" doing these things.

An AI capable of recursive self-improvement isn't allowed by the EU AI act, for example. But perhaps more seriously, You have it backwards: people without access to such expensive equipment are more incentivized to go the self-improving route. Your ideas about "millions" being necessary might be far off?

You entirely discount human stupidity and lack of imagination. Humans are already being replaced with AI, not because AI was strictly better, just because it's cheaper.

▲

AlexErrant an hour ago | parent | next [-]

Are these incentivized humans as organized, well-funded, or smart as the people working at the frontier labs?

If my "millions" is an underestimate, why haven't other labs using their own unique training methods/data/etc stumbled into RSI? Sorry if I'm misunderstanding; I'm struggling to understand what you wrote.

I'm pretty sure we agree on humans being stupid, but that doesn't mean that suddenly we get human extinction. You gotta connect the dots for me here.

	▲	TedDoesntTalk 24 minutes ago \| parent [-]
		Not OP. Why would it be millions in 50 years? The think about nuclear weapons. In the early days, it was limited to the super powers. Now 9 countries have them and a country like Iran is capable of acquiring them. Is destructive AI be any different? Genuine question.

▲

Retric an hour ago | parent | prev [-]

Self improving AI runs into the same issue as prefect comprehension, you can’t get arbitrarily better at everything.

The idea AI can get better at everything at the same time is a holdover from deeply flawed science fiction not some realistic goal.

	▲	Lerc 37 minutes ago \| parent \| next [-]
		This can be generalised to the curve plotting of the singularity itself. If the time between advances is a + b and a is the proportion of the period that can be improved by advances then you won't reduce to a gap of nothing between advances, you reduce to a gap of b. Assume the invention of the plow and the invention of the sword is 500,100 units and a was the 500,000, you wouldn't even know the 100 as in there. Maybe we're at a=2000 now and b is still siting at 100. Assuming we'll reach infinity because we're dividing by the only variable we see and it is decreasing in size seems nuts if the reason we might not see other variables is because of the size of the variable we can see.
	▲	afthonos 41 minutes ago \| parent \| prev [-]
		Even if you’re right, that doesn’t mean AI can’t get better than humans at everything.