> But if you told them about social media, I think the story would be different.

It would be utopian, like how people thought of social media in the oughts. It's a common pattern through human history. People lack the imagination to think of unintended side effects. Nuclear physics leading to nuclear weapons. Trains leading to more efficient genocide. Media distribution and printing press leading to new types of propaganda and autocracies. Oil leading to global warming. IT leading to easy surveillance. Communism leading to famine.

Some of that utopianism is wilful, created by the people with a self-interested motive in seeing that narrative become dominant. But most of it is just a lack of imagination. Policymakers taking the path of local least resistance, seeking to locally (in a temporal sense) appease, avoiding high-risk high-reward policy gambits that do not advance their local political ambitions. People being satisfied with easy just-so stories rather than humility and a recognition of the complexity and inherent uncertainty of reality.

AI, and especially ASI, will probably be the same. The material upsides are obvious. The downsides harder to imagine and more speculative. Most likely, society will be presented with a fait accompli at a future date, where once the downsides are crystallized and real, it's already too late.

▲ godelski 19 hours ago | parent | next [-]

  > It would be utopian

People wrote about this. We know the answer! I stated this, so I'm caught off guard as it seems you are responding to someone else, but at the same time, to me.

London Times, The Naked Sun, Neuromancer, The Sockwave Rider, Stand on Zanzibar, or The Machine Stops. These all have varying degrees of ideas that would remind you of social media today.

Are they all utopian?

You're right, the downsides are harder to imagine. Yet, it has been done. I'd also argue that it is the duty of any engineer. It is so easy to make weapons of destruction while getting caught up in the potential benefits and the interesting problems being solved. Evil is not solely created by evil. Often, evil is created by good men trying to do good. If only doing good was easy, then we'd have so much more good. But we're human. We chose to be engineers, to take on these problems. To take on challenging tasks. We like to gloat about how smart we are? (We all do, let's admit it. I'm not going to deny it) But I'll just leave with a quote: "We choose to go to the Moon in this decade and do the other things not because they are easy, but because they are hard"

▲ cwnyth 19 hours ago | parent | prev [-]

All of this is a pretty ignorant take on history. You don't think those who worked on the Manhattan Project knew the deadly potential of the atom bomb? And Communism didn't lead to famine - Soviet and Maoist policies did. Communism was immaterial to that. And it has nothing to do with utopianism. Trains were utopian? Really? It's just that new technology can be used for good things or bad things, and this goes back to when Grog invented the club. It's has zero bearing on this discussion.

Your ending sentence is certainly correct: we aren't imagining the effects of AI enough, but all of your examples are not only unconvincing, they're easy ways to ignore what downsides of AI there might be. People can easily point to how trains have done a net positive in the world and walk away from your argument thinking AI is going to do the same.

	▲	godelski 18 hours ago \| parent \| next [-]
		`> You don't think those who worked on the Manhattan Project knew the deadly potential of the atom bomb?` I think you have missed an important part of history. That era changed physics. That era changed physicists. It was a critical turning point. Many of those people got lost in the work. The thrill of discovery, combined with the fear of war and an enemy as big as imagination. Many of those who built the bomb became some of the strongest opponents. They were blinded by their passion. They were blinded by their fears. But once the bomb was built, once the bomb was dropped, it was hard to stay blind. I say that this changed physicists, because you can't get a university degree without learning about this. They talk about the skeletons in the closet. They talk about how easy it is to fool yourself. Maybe it was the war and the power of the atom. Maybe it was the complexity of "new physics". Maybe it happened because the combination. But what I can tell you, is that it became a very important lesson. One that no one wants to repeat: it is not through malice, but through passion and fear that weapons of mass destruction are made.
	▲	energy123 19 hours ago \| parent \| prev [-]
		> You don't think those who worked on the Manhattan Project knew the deadly potential of the atom bomb? They did. I am talking about the physicists who preceded these particular physicists. > And Communism didn't lead to famine - Soviet and Maoist policies did. Communism was immaterial to that. The particular brand of agrarian communism and agricultural collectivization resulting from this subtype of communism did directly cause famine. The utopian revolutionaries did not predict this outcome before hand. > People can easily point to how trains have done a net positive in the world and walk away from your argument thinking AI is going to do the same. But that is one plausible outcome. Overall a net good, but with significant unintended consequences and high potential for misuse that is not easily predictable to people working on the technology today.