The argument was fairly specific: there is a groupthink involved, and so the enterprise is suspect.

It’s an excuse not to think, not an argument.

The doomer argument is straightforward: a powerful AI will have goals. If those goals don’t include the welfare of humanity, a sufficiently powerful AI will kill us either by accident, by indifference, or intentionally when we get in its way. Making an AI that includes the welfare of humanity in its goals is incredibly hard.

That’s it. To refute the doomer argument, you can refute any part of that. I notice that none of the people bringing up fanfics, cults, and weirdness are doing that. No, they’re instead trying to convince people that doomers are uncool. Since only cool people can know things, QED.

▲

disgruntledphd2 14 hours ago | parent | next [-]

> a sufficiently powerful AI

We don't have this kind of AI, and (as of right now), unless humans link the AIs we do have up to a bunch of killer robots (i.e. drones) then this is vanishingly unlikely.

Like, any scenario where AI kills us all requires us to give it the tools that it needs to do this (as well as a desire to do this, as right now they just attempt to generate text to solve a problem/question they are posed).

	▲	afthonos 13 hours ago \| parent [-]
		No one is saying we have it, they’re saying they’re building it. And what mechanism to control these drones, which are autonomous or remote-controlled machines, do you imagine that humans will be able to use, but AI won’t? And what in the history of the past five years, gives you confidence that not a single human will give the AI the access it needs to do bad things? Incompetence alone is sufficient to kill us, let alone humans that genuinely, or confusedly, want bad things.

▲

ribs 6 hours ago | parent | prev [-]

It’s reasonable to cast doubt upon the fitness of ideas based on their sources, particularly when the ideas are hard to adjudicate. It reasonable to filter what one engages with partially based on the sources.

It would be foolish to ignore sources.

	▲	afthonos 2 hours ago \| parent [-]
		I agree on complicated ideas, and on facts (because obviously different sources will selectively report facts, even if they’re trying to be fair). The ideas here are not complicated. The argument is three sentences.