Remix.run Logo
fc417fc802 6 hours ago

I think you've misunderstood me. I wasn't arguing against risk in any general sense - quite the opposite.

My only position is that "and then the AGI killed everyone" is so oversimplified that it makes children's bedtime stories look sophisticated.

Physical hardware is relevant because unlike computer programs results aren't instant. The supply of materials, manufacturer, and deployment are all slow processes. Opponents generally have time to react, and if AGI is an opponent then I think it's safe to assume that it also has to contend with other AGI.

I am unconvinced that there should be a clearly delineated divide with all AGI on one side and all humans on the other.

I agree with you about the myriad non-combat risks but again I think the prediction is one of hopeless complexity. For every problem that's inadvertently created a solution can probably be arrived at to patch things up. It's not unlike the present, just faster. Break things, fix them, but the fix breaks other things, repeat ad nauseum. Hopefully we survive the process.