Prediction: AI ends humanity not via some super cool/scary/robopocalypse - but as a marketing stunt gone wrong when a Frontier LLM accidentally knocks out water/electric/gas by hacking in and trying to patch them.

Phillip K. Dick meets Idiocracy.

▲

bluGill 21 minutes ago | parent | next [-]

That won't end humanity. My propane generator from 1980 still works and I have plenty of fuel in a tank. This is enough to cold start generation of the grid. (the power company won't need my generator - they have their own plans to cold start, but worst case they can fall back to mine). While a hacker unstopped could do a lot of damage, most of the parts will still be functional and we have more than enough engineers to figure out how to bypass the areas they can take out. It might mean a few years of power rationing while the parts AI can break are repaired, but we can restart.

▲

utopiah an hour ago | parent | prev | next [-]

I remember (sorry can't give a proper quote) a biologist interviews about the fear of lab grown "super" seeds escaping the lab. They chuckled basically saying that nature is a very VERY challenging place. There are plenty of ecological niches but they are well guarded by incumbents.

Sure some new hacks will take place, including on poorly guarded infrastructure and yes it will have some very unfortunate consequences... but also infrastructure is precisely designed to be resilient. There is quite a bit of failsafe, redundancy, etc built in which is precisely why those projects are typically slow and expensive, unlike a random website for a restaurant.

TL;DR: nope, some isolated incidents will happen but without chain reactions.

▲

karmelapple 24 minutes ago | parent [-]

Biologists take ethics much more seriously than any computer engineer or computer scientist that I've seen.

I saw a very notable scientist talk about topics like this and how they wrestle with them. Things like a "gene drive" [0] are being experimented with in the world [1], and biologists think very hard before doing certain things with the powers they wield. And I have not heard a life scientist laugh off ethical or spreading concerns when a thoughtful question was asked... but maybe the interview you're thinking of was a more lighthearted one?

0. https://en.wikipedia.org/wiki/Gene_drive 1. https://www.science.org/content/article/controversial-gene-d...

	▲	utopiah 12 minutes ago \| parent [-]
		My point isn't so much about the work ethics of one or another but rather than systems, even interconnected ones, are rarely as genuinely interdependent as science fiction drama lead us to believe. Yes there can be terrible catastrophes e.g Fukushima, but what actually happened, within which extremely rare conditions, versus what we freely imagined is quite different. We might need more reminders of those risks to properly insure such systems don't lead to the collapse of one to another (sadly) but still we manage to learn from it.

▲

bix6 an hour ago | parent | prev [-]

OpenAI and Anthropic vs Brawndo Corporation. The final showdown.