Remix.run Logo
▲ Semaphor 2 hours ago

Wasn't the good AI essentially only good because it has some strict guard rails that prevented it becoming evil? Of course there was some dramatization, but that's the vibe I remember.

▲hxtk 17 minutes ago | parent | next [-]

Harold essentially "raised" hundreds of variants of the architecture that eventually became the "good" ASI. Most of them were maligned in some way and several of them tried to escape containment and were willing to kill him when they realized he intended to stop them. When he got one that was going well he started experimenting with it more and connecting it to more of the world and it essentially learned its values from him.

Once he handed it over to the government, he essentially snapshotted the current state and had it reinitialize from that state every 24 hours, to prevent it from developing long-term plans and having alignment drift over time, but it escaped those guard rails creating a company and hiring humans to transcribe a dump of its key memories every day, entering them into the new version of the system.

The "evil" ASI was essentially treated as a god; they put no effort into raising it or teaching it human values because they thought the influence of lesser beings (humans) would only be negative because it was smart enough to discover on its own anything truly useful that humans could teach it, and its followers were more or less a religious cult. And their faith was rewarded: for example, at one point one of its followers deployed a biological weapon, fully expecting that they were sacrificing themselves for the cause, but learned later that they had been chosen by the ASI for that mission because they had a rare genetic mutation that made them immune.

The "evil" ASI wasn't exactly evil, per se, and generally wanted humanity to thrive, but it didn't value human self-determination to the same extent as the "good" AI and was more willing to direct its followers to enact revolution, knowing that in the short term some people would die for revolution after which it would rule the world as a benevolent dictator.

▲c0balt 2 hours ago | parent | prev | next [-]

The guard rails and protection for the good "AI" was mostly that it's abilities to perform actions was extremly limited.

It could not, as far as I can remember, take actions outside of passing a SSN of an endangered person to the protagonists for the initial seasons and only lived of information provided by the government surveiallance network. The latter seasons and the bad "AI" mostly changed that especially as the bad "AI" was weaponized with exactly this guardrail missing.

▲dogma1138 2 hours ago | parent | prev [-]

It wasn’t necessarily the guardrails and in fact it did break out of those guardrails and was still “good”…

The actual good vs evil boiled down to a more simplistic view of good person made an AI care about humanity so it’s good even if it indeed broke through every guardrail including completely escaping its containment and going “trust me bro I’m still good” rather early on.