The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful.

If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.

▲

andai 10 hours ago | parent | next [-]

Well, at that point it doesn't even need to be a virus. It can just be nanobots that atomize your liver. The substrate is a mere technicality.

	▲	jeremyjh 10 hours ago \| parent [-]
		Right. A superintelligence will kill us in one or more of thousands of possible ways for thousands of possible reasons, perhaps none of which we may understand. Short of a superintelligence - the difficulties of viral bioweapon engineering the author points out are quite meaningful.

▲

cobbzilla 11 hours ago | parent | prev | next [-]

please define fully general superintelligence & current state of alignment

is this a tautology?

	▲	andai 10 hours ago \| parent [-]
		Not the guy but, alignment seems to be several decades away, while self-improvement seems to be several years away. (Also you don't even need superintelligence to do a lot of damage. In fact you don't need any intelligence!)

▲

11 hours ago | parent | prev [-]

[deleted]