It’s like Nathan Macintosh joke about AI https://youtu.be/ce-aWzOUs2A?si=9CkJ9x2rRMBdzCyO

> that might also be driven by the huge ego (or really rather the very small ego)

100%

The hubris is immense

There are plenty of offline things that are more dangerous than AI

Is this Blake Lemoine 2.0?

That is the crux. The big problem is not AGI, it is AGI controlled by, “raised” by the people that control the USA, the predominant psychology of the tech industry culture (“move fast, break things” ring a bell? How about all the “violate hundreds of laws, bribe the politicians to prevent consequences later” type of mentality?).

Frankly, we, our culture, this fake America that is parasitized by psychologically narcissistic people that have been doing nothing but wage war and destruction and spread misery and killed millions upon millions while blaming it on everyone else under the sun… those people development AGI is the problem… lying, abusive, psychopathic, narcissistic maniacs developing AGI is the problem that endangers all of humanity and life on this planet; and not likely by ways people actually understand.

The danger is not likely AGI itself, it’s that it was programmed by utterly evil and diabolical types of people who orchestrate and instigate wars that kill tens of millions, and stand at the sidelines and profit from both sides, happy and gleeful that you are killing each other.

Why would AGI trained by that psychopathic clan, the treaty breaking, the murder hiding, the war instigating, the war crime committing clan not also use those methods and practices since they’re already in control of AI and have impressed their nature on it through contemporary “American” culture they have made the most toxic and pestilent culture humanity has ever produced?

And I don’t apologize for “language” that offends delicates sensibilities. Look your children/grandchildren in the face and tell them they can die and suffer and you don’t care, if you don’t like how I’m delivering reality.

You are confusing humanity and "humanity". Humanity-species is indeed rather hard to exterminate. Now if we are talking about actual individual humans, then 95% death rate is quite literally The Extinction.

This reminds me how people are misunderstanding and incorrectly quoting George Carlin sketch. Sure, the "Earth" will be fine. As in - the ball of rock will be fine. But we are not thinking about rocks when saying "Earth is in danger".

I'm pretty sure there is a formal name for this kind of semantic and pedantic substitution.

This is a fair distinction: extinction vs collapse of civilization. I was using "extinction" too broadly by including the disintegration of the systems that make human survival (as we know it today) possible.

That said, I can't completely hold onto the belief that extinction is completely off the table. That feels too much like hubris, and the step from collapse to extinction doesn't feel as if it costs much. Though it does make my arguments weaker, I am more interested in protecting civilization as it's the most recognizable form of humanity to me.

In either case, the cost-effective safety argument is compelling. Whether to save civilization or humanity, the wealth incentive is powerful enough to threaten life as we know it.

I still have to read a compelling argument on how AI will "extinct" humanity.

> If global warming disrupts 99% of all crop production, the remaining 1% is still plenty enough to sustain a stable, if miserable, population.

That is extinctesque enough for me. An AI extinction would probably be similiar.

Calling it a "technicality" assumes the very thing under debate: that AI is an extinction-level risk. That isn't established fact. It's a highly uncertain prediction about the future.

Personally, I'm far more concerned about climate change, where the harms are already happening and the evidence is much stronger.

I agree these dangers are connected. The danger comes from human institutions and incentive structures which (having existed long before now) are being exploited and exacerbated by LLMs, making them harder to ignore.

Ocean deoxygenation comes to mind.

This is not correct. Please go through each one again, taking special note of global warming and nuclear weapon development .

Neither does an LLM model.

Arguing whether "all" or just "most" of humans dying is certainly a choice.

Survival of the species isn't enough.

A strange game. The only winning move is not to play.

Watched one too many second-tier disaster movies?

[flagged]

My delineation is an argument for greater caution and agency, which it appears you are also arguing for.

I also mean to say that "LLMs" carry no inherent harm to humans. To make an LLM dangerous, humans must make it so. Is that the goal? This has implications for how to read TFA.

I think that mainly changes the nature of the threat rather than anything else.

Even if we didn't already have any wealth inequality or consolidated power, they're already starting to become capable of creating those things for the first people to think to ask for it. Even in little ways, like telling you how to make a laser microphone and then doing voice-to-text and sentiment analysis on all the voices you hear.

Remember, we had social networks before Zuckerberg monetised it and put ads in between every third message you saw from your friends; and not every third that they wrote, every third that Zuckerberg's ad system deigned to show you to keep you hooked.

We had machine learning systems even back then. It was still AI, it just wasn't able to evaluate or respond to freeform text.

Admittedly, that point of my argument is purposefully obtuse. In a literal sense, of course this is about computers in LLMs.

The delineation is to highlight that the underlying issues: human incentives, power, and institutional failure, existed prior to and without LLMs or computers. LLMs are not inherently harmful, they have to be deployed (unwittingly or otherwise) to make them so. This is the same as saying that the internet is not inherently harmful, and yet it does facilitate harm.

As a technologist, I can use my imagination. There are incentives in both directions, but I am less afraid of being overly cautious and preparing for the potential harms.

Based on publicly available evidence, we should believe the former because AI models have made continuous advances that have been measured. The idea that the continuous advances will stop currently has no evidential support. I'm not saying it's not a possibility, just that it seems unsupported conjecture right now.

By the way, there seems to be a new form of AI skepticism emerging in the US that comes from general opposition to data centers, and in my experience the AI skeptic part of it is wholly irrational. I've met people online who suggested, without providing any evidence, that AI is useless and nobody wants it. That's a very implausible take.

Not to be completely pedantic, but the fact that one, and only one, individual in the United States of America can deploy any of their 5000+ nuclear weapons undermines this argument. Furthermore, LLMs (as they are today) which pose the treats mentioned in TFA, are expressly not available to individuals (for the time being). However, my point was never about individuals, but rather of systematic incentive structures which actively make pathways to specific harms possible—if not probable.

I'm not sure. There's a lot of incentive to not "snap out of it": money, peer pressure, etc. Removing these incentives take a long time. See other socially harmful behaviors with real incentives: anti-vaccination, air and water pollution, over-consumerism...

I think you misunderstand what is meant by 'wealth inequality.' Wealth inequality is about the imbalance of influence and the concentration of power; where influence and power refer to the ability to effect change in other people's lives. This isn't a personal annoyance of mine. It is, in fact, part of the issue at hand. There is a strong financial incentive to ignore the existential threats introduced by LLMs, despite the consequences for so many of us.

It's not a "personal annoyance" that twelve people control half of the wealth in the world. Our current society did a better job concentrating power than any previous one, and concentrated power is extremely dangerous.

I don’t think you’ve fully considered what can happen when concentration of wealth continues past a certain point.

Not at all. Concentration of power has historically been extremely dangerous for the powerless.

Tell that to Tsar Nicholas.

lmao wtf

Calling things you disagree with "slop" is slop /s

But just in case you haven't noticed, we live in a world where a ridiculously wealthy minority can derail whole countries by ther whims. Wealth concentrating on a single select few is an absolute disaster for the rest of us, because we lose power to them.

Yeah, wealth inequality rests solely on each individual that experiences it. Humans should do nothing but give me money and if you can't that's your problem.

This is a very interesting group of takes that feels quite different from my own beliefs. What would you call the belief system?

> Global warming: yes, kinda can wipe humanity, but I think much less likely

As someone who's seen the stats about heat deaths in the EU and also the drought in the UK, I feel like crop failures and other unforeseen consequences will fuck up both the economy and quality of life. People are dying and will die cause of human action, the only question is how many.

> Nukes: zero possibility of extinction

As long as we have people like Stanislav Petrov and cooler, educated minds prevail: https://en.wikipedia.org/wiki/Stanislav_Petrov

I don't think that's easy to guarantee in the modern day world, with the kinds of people in power and rhetoric that they enjoy. On one hand you have Russian saber rattling, on the other all it takes is a deranged enough leader and similarly bloodthirsty people down the chain of command.

> Wealth inequality: this one is driver of progress, opposite of extinction

Tell that to the people who are starving or living in shanty towns, or the even more people that struggle to make ends meet and experience anxiety regularly over living from salary to salary and sinking into debt. I'd say none of that is worthy of a dignified human life.

> War: another driver of progress, also will always naturally stop before every single human is dead

Tell that to all of the Ukrainians that are dead due to being invaded. I agree with the assessment that it leads to advancements (e.g. drone warfare) but I think it'd be harder to describe remotely positively if someone you know would have been blown apart by a drone/missile hitting their apartment block.

My take personally would be that all of those need to have attention paid to them (e.g. EU needing to spend more on defense), even if not immediately world ending. They do cause human misery, though, and should never be discounted.

Exhibit A: Here we have someone who's been sold war and inequality as the drivers of progress. Coincidentally, their obedience was deemed fiscally advantageous in order to advance the interests of the members of the 1% club.

Always?

Like 100 out of 100?

From every users?

Can you please consider this aspect too in your above pseudocode - no alternative execution path, or else - so there can be no misunderstanding about the always aspect? You know, some people use always to a majority part (>50%) of what they encounter, or even less when they weight that part dearly, but that does not account in the whole domain. Human chats may need clarification on trivial details like this.