Okay, so what should we do?

I for one would like nobody to build the AI, is that an option? How do we get it on the menu?

Should we trust the people who say "nothing could possibly go wrong and there's no reason to worry or think about safety measures, if there are any trifling problems along the way we'll just figure it out as we go?"

▲

pdonis a day ago | parent [-]

I don't think nobody building AI is an option, since we can't control what China and Russia do.

I personally am not concerned about that as far as an existential threat goes, because I don't think AI, in itself, poses one; I think it will hit diminishing returns. But of course it is still capable of serious harm when humans misuse it.

The best analogy we have is probably weapons of mass destruction--we have international treaties about them and we don't let just anybody have them. That's the kind of framework I think we need, and it goes beyond "regulation" in the ordinary sense.

Use of such weapons is considered by international treaty to be an act of war; use of AI by, say, China and Russia against our infrastructure should be the subject of a similar international treaty with the understanding that it would have similar consequences.

Domestically, humans who misuse AI to do serious harm could be treated like, say, the people who tried to spread anthrax years back.

▲

mitthrowaway2 a day ago | parent [-]

You seem focused on intentional misuse but I think there's an equal or greater potential for harm happening by accident without intention. Nuclear weapons, at least, have so far reliably and predictably done exactly as their owners intended. Nuclear power plants on the other hand have, on occasion, blown up despite nobody intending them to do so, causing some populated areas to become uninhabitable.

Don't you think that AI has by nature at least a little more potential for unpredictable results and unexpected harms than your typical technology? It seems like a major oversight to completely disregard this aspect.

We can maybe hold someone responsible when things go wrong and harm is done, and blame them for negligence as though the harm were intentional, but from a preventative safety perspective that's not usually sufficient nor even always helpful for preventing accidents.

▲

pdonis a day ago | parent [-]

> I think there's an equal or greater potential for harm happening by accident without intention.

The WMD analogy doesn't quite cover this, yes. Maybe a better analogy here would be toxic waste. Yes, someone might not have intentionally set up a factory that puts dioxin into the water, but they're still responsible. Which means they need to take precautions to make sure such accidents can't happen. Similarly, those who build AI need to take precautions to make sure the AI they build doesn't accidentally cause serious harm.

	▲	mitthrowaway2 6 hours ago \| parent [-]
		> Similarly, those who build AI need to take precautions to make sure the AI they build doesn't accidentally cause serious harm. I am ten thousand percent in agreement that this is needed. And yet, I don't know how we ensure and incentivize that these precautions are taken by the actors involved, and as I understand it, even many of the top researchers in the field agree that they don't know what precautionary measures would even be effective, let alone sufficient. It seems that game theory has so far been pushing the AI companies to build it anyway without a robust solution for preventing harms, even while they express worry in public about the potential for those harms. Again we can hold corporations or people responsible for damages after the fact, but many examples can be cited to show how that tends to be insufficient.