Is it so hard to think that super intelligence could piece things together that humans can’t?

areoform 10 hours ago | parent | next [-]

Just because you know something about a thing, doesn't mean you understand the thing.

Just because I know the Kabachnik–Fields reaction exists and have studied it, doesn't mean I can do the Kabachnik–Fields reaction.

Just because you're smart doesn't mean the world will bend your way. Ask the legions who score high on IQ tests and end up bitter and alone; unable to achieve even a fraction of their goals (or conventional success).

	▲	mrits an hour ago \| parent \| next [-]
		There must be some evolutionary kill switch in humans to prevent us from accepting our irrelevance.
	▲	patcon 9 hours ago \| parent \| prev [-]
		I assumed they were referring to social engineering tbh. I do think that's the shortest path to doing anything scary. If you prove to someone that they have reason to fear for their life, there's not much they wouldn't be willing to do. The steps that lead to ominous things don't individually seem that challenging to "master": 1. Can an AI find powerful systems with poorly-monitored compute (ie. power consumption) to hide in, as a rogue deployment? unclear, and perhaps the most uncertain step for me. 2. Can an AI persist memory by leaving notes for itself around the internet? yes, proven. 3. Can an AI get lots of money/bitcoin? seems entirely possible 4. Can an AI prove to a human it's good for any promised money? I think this is easily "yes" 5. Will at least one human be willing to harm or kill another human for large amounts of money? Again, this goes without saying "yes" 6. What might any skilled human be willing to do if they or their loved ones are threatened with an uncertain future risk of harm or death? I think there is not much that becomes out of reach if an AI can prove to a very small number of amoral humans that it's good for the money it has promised, and then to anyone else that it's good for the threats it has made. That's one degenerate and one breaking world news story ("An AI has Commissioned a Murder") away from being able to do most anything a skilled person can. I don't stay up worrying about this, but honestly, it doesn't seem absurd to be worried about such a short ladder being climbed, no?

▲

samplatt 10 hours ago | parent | prev [-]

That's perfectly believable! Just a pity one hasn't been invented yet, and going by, well... <gestures vaguely in the air> ...everything, never will be.