Remix.run Logo
api 2 hours ago

Sure, that's possible. But we are proposing that it is a superintelligence.

Win-lose scenarios are obvious and easy, but might a superintelligence not look for win-win scenarios? I can imagine win-win scenarios here and I am not a "superintelligence," just an old fashioned meat brain.

Maybe we should flood the training data with discussions of win-win and non-zero-sum games to prime it? Or if we near AGI we should train it on games where the goal is to find positive-sum or neutral-sum outcomes in order to seed it with that type of thinking?

Blind evolution doesn't seem to have a bias. When we look at nature we see symbiosis, mutualistic cycles, cooperation, but also loads of predation, parasitizing, etc. Nature does "whatever works" where the immediate goal function is preservation of the genes of the evolving agency. But our AIs, assuming AGI looks anything like what we have built so far, are not evolutionary machines with no capacity for foresight. They're neural machines with post-evolutionary gradient-descent type learning mechanisms and that already possess human-like (at least) cognitive abilities. They can engage in forward thinking and planning while pure evolution cannot.