Remix.run Logo
▲ origini 2 hours ago

Even in normal jobs we have such a HUGE difference between two people. The only organization which is working against are pilots with procedures and checklists.

And every year a new human comes along and has to learn all of it again.

Of course ONE AI agent and teaching ONE AI is massivly in an advantage as soon as its as good as the avg person in the job and it blows everyone else out of the water as soon as it gets a tag better.

▲joshAg an hour ago | parent | next [-]

There are other professions with procedures and checklists. Just off the top of my head there's safe surgical checklists to minimize errors during surgeries [1] and checklists for A+P procedures on planes, checklists for operating/working at nuclear plants, and for factories/manufacturing plants making precision parts or operating within a good safety regulatory regime (like semiconductor fabs or auto manufacturing), to say nothing of checklists in the military, which designs equipment to have easily disableable safety mechanisms if it has any at all because the operator might need to operate it outside normal, safe limits [2].

[1]: https://www.who.int/teams/integrated-health-services/patient... [2]: https://en.wikipedia.org/wiki/Battleshort

▲KolmogorovComp an hour ago | parent [-]

But all these procedures stem from pilot checklists.

▲lixtra an hour ago | parent [-]

Or from elections.[1]

[1] https://www.solist.blog/pilots-did-not-invent-checklists-in-...

▲lazide 2 hours ago | parent | prev | next [-]

Except that ‘one AI’ will have it’s own blindspots which can be exploited universally and all at once, until the magic fix goes into it.

It’s literally a monoculture.

For example, if an iteration of the singular AI ends up being crashable with a magic visual pattern or QR code (highly likely at some point), that is a major national security risk if that AI is piloting aircraft, as the entire nations aircraft could be taken down by anyone projecting said image in it’s visual plane until the fix gets pushed.

And these systems are far too complicated to exhaustively test all possible inputs before release.

There is a reason why nature rarely has such monocultures - that live for very long, anyway.

▲onefiftymike 33 minutes ago | parent | next [-]

Can't some of this be mitigated by having voting systems? Have three AI systems from three different vendors and they vote on the solution? Then any one failure will just fail the one system. The chance of three identical vulnerabilities in three different architectures is pretty slim. You can also have some of the Airbus "watchdog" hard coded oversight to prevent flying outside a defined envelope for most normal flight.

▲origini an hour ago | parent | prev [-]

With RL it will learn from all misstakes once while with humans after a generation, information might just be gone.

Its a possible as an attack vector for sure.

But in comparision to nature, compute can speedrun all of this with RL.

▲disgruntledphd2 an hour ago | parent [-]

RL data is really really really expensive to collect and train on, and only works for verifiable tasks, which is not most tasks.

Even a lot of what software professionals do isn't verifiable.

▲datsci_est_2015 36 minutes ago | parent | prev [-]

> Of course ONE AI agent and teaching ONE AI

One of the most misinformed things I’ve read on Hacker News recently. That’s not how any of this works.

Has the average quality of discussion plummeted since the advent of LLMs and every tech bro starting to comment wildly out of their depth?

▲origini a minute ago | parent [-]

I'm reading about AI, LLMs and ML for over 4-8 years now and finetune models etc.

How about you are less snarky and try to be clear what your problem with my statement is?

Should I say one model/one model family? etc.?