When you can't tell if it's profit motive or the AGI has already taken over.

IMO, capitalism is itself an unaligned artificial intelligence, with its own logic and goals orthogonal to human life. All “ex risk” paranoia is a projection of sublimated anxiety about capitalism.

It's paperclip maximization either way.