Remix.run Logo
▲ lazide 2 hours ago

Except that ‘one AI’ will have it’s own blindspots which can be exploited universally and all at once, until the magic fix goes into it.

It’s literally a monoculture.

For example, if an iteration of the singular AI ends up being crashable with a magic visual pattern or QR code (highly likely at some point), that is a major national security risk if that AI is piloting aircraft, as the entire nations aircraft could be taken down by anyone projecting said image in it’s visual plane until the fix gets pushed.

And these systems are far too complicated to exhaustively test all possible inputs before release.

There is a reason why nature rarely has such monocultures - that live for very long, anyway.

▲onefiftymike 34 minutes ago | parent | next [-]

Can't some of this be mitigated by having voting systems? Have three AI systems from three different vendors and they vote on the solution? Then any one failure will just fail the one system. The chance of three identical vulnerabilities in three different architectures is pretty slim. You can also have some of the Airbus "watchdog" hard coded oversight to prevent flying outside a defined envelope for most normal flight.

▲origini an hour ago | parent | prev [-]

With RL it will learn from all misstakes once while with humans after a generation, information might just be gone.

Its a possible as an attack vector for sure.

But in comparision to nature, compute can speedrun all of this with RL.

▲disgruntledphd2 an hour ago | parent [-]

RL data is really really really expensive to collect and train on, and only works for verifiable tasks, which is not most tasks.

Even a lot of what software professionals do isn't verifiable.