> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming.

I really think we need to stop giving credence to people who have

1) Been consistently wrong with all their predictions

2) Demonstrated an endless spiteful cynicism

Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk outside them. You don't want the society they want, and they things they believe in are unlikely to happen.

It would be far, far better to listen to the people who never fell down every misanthropic rabbit hole, rather than the ones who have noticed it this time, but want you to still believe them on every other topic.

If you replace "AI" with "Adonai" in EY's framing, it reduces to biblical parables from his childhood about arguing with God to negotiate a new covenant.

He's clearly a bright guy, but a lot of his work seems to be trying to reconcile Old Testament narrative patterns with atheism, and simply slotting an omnipotent AI in as a replacement.

The guy was saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility. How is that in any way "consistently wrong with all their predictions"?

"Consistently wrong" seems a bit much. Seems like being directionally right early that AI would be a Big Deal and scary should count for something? It doesn't mean any details or other predictions are right, though.

You can criticize being wrong, but why is the doomer argument "misanthropic" or "malevolent"?

> they things they believe in are unlikely to happen.

Sorry, but most times I've talked to someone who says this (about AI completely replacing humanity), they don't have anything to say about why it's unlikely to happen other than:

- "well, it's just such an extreme outcome, it must be improbable"

- "humanity has survived near scrapes with extinction before"/"all the previous doomsday predictions have been wrong"

There is a cliche that all teenagers deep down believe that they are invincible. It seems to me that humanity is still a teenager in this respect: We don't take seriously the possibility of our own extinction. While one might think that the invention of nuclear weapons would serve as a wake-up call, if anything it has done the opposite.

I'm willing to hear arguments besides the two above, if you have them. (And to be clear, being replaced by AI doesn't necessarily mean being replaced by LLMs in particular. They are a relatively new development.)

He talks about AI cutting through popups, but if you follow that line of thinking further, screens and traditional websites likely fade in importance. It becomes more talking and texting to AI and visuals on the topic at hand appear in smart glasses, lock-screen–centric AI phones (where website visits dwindle), digital picture frames, TVs, etc.

I sorta gather (just a hunch) this is the type of device Open AI is working on.

Though if there are no websites anymore or ones not many visit how would AI stay relevant? I wrote some thoughts about this in early June https://ryanspahn.substack.com/p/ai-to-pay-for-all-americans... that AI needs to pay it's fair share for all it has taken and all it will continue to take from us. A symbiotic relationship needs to be established!

The cynicism and misanthropy is nonsense and a direct result of the internet/social media landscape. It's BS.

In the current media ecosystem, what gets you "social cache" or whatever? Let's call them "cool points." Well, right now, the current metric is basic "likes" or "retweens" or "upvotes" or whatever. And to get those, you have to make a claim and then that claim has to be evaluated by others. But the evaluation by others is not really reality, it's just what others think, and it's based on outcomes.

This gives us 4 possible prediction-outcome scores.

You predict DOOOOOOOM!, and the outcome is DOOOOOOM! - in this case you look like a genius because you predicted doom

You predict DOOOOOOM!, and the outcome is "ok, or even great!" - nobody cares you got it wrong because things are fine

You predict UTOPIA!, and the outcome is "ok, or even great!" - people say, "wow that was cool, you got it right" but its not that great of a reward from your social group (the internet) because things were fine

You predict UTOPIA!, and it turns out DOOOOOM! - you look like a moron.

Now looking like a moron is is wayyyyyy worse than what happens if you get it right. Indeed, unless you are really damn sure of utopia, you have a social incentive to predict DOOOOOM! that's greater than the alternative. You can predict DOOOOOM! 100 times and get it wrong 99 of them and nobody cares because that one time you get it right you look like a genius. It's a huge huge problem in the current media landscape and it's why everything is killer robots right now and not "hey we could build a star trek utopia!" It's effectively a selfish NIMBY-ist view of the world in a way, "why build anything, it's just going to ruin everything!" It's a pessimism that literally desires the end of humanity to "prevent the likely future sufferings." It's a simplistic view of a complicated world, and the rewards that go to "DOOOOM!" don't match much with reality.

That's why the internet is so doom pilled, that's why everyone is a cynic, that's why everyone is kind of an asocial asshole about new ideas on the internet and even in your friend space.

You guys know you can just "do stuff?" I mean, yeah it takes money and time, but you can allocate that - especially now that you don't have to grind out css for hours and hours. But you can just do things - whatever you want. You've always been able to, but it just got a lot easier in many many domains, and it's going to get even easier than that.

People are going to scream slop and "that sucks" until the cows come home, because that's what is rewarded on the internet these days. Meanwhile, as people we MUST believe that we still have some agency to do stuff, then go and actually do stuff while the haters are sitting on their couches shellacking each other with misanthropy.

And don't even get me started on the loudest voices in the ecosystem right now.

Like, Yudkowsky is a nobody. Not to be a dick, but he's a high school dropout forum guy who got lucky. I'm not saying he's always wrong, or even that I oppose autodidact stuff - indeed I'm always self teaching. But this guy never took a ML class in his life, I doubt he can solve an integral un-aided, I imagine some people here have seen this: https://www.reddit.com/r/badmathematics/comments/2bazyc/comm...

And we're all deferring to that guy like he knows wtf he's talking about? It's the AI equivalent of the self-educated anti-vax mom at the height of the pandemic.

And he's not alone either, there are a lot of people who are uninformed, unknowledgeable, and confidently wrong on the internet desperately trying to peel off your attention for an instant so you can look at their site, their ads, and their content.

The same sorts of things can be said for the Zitrons of the world. He's a hell of an entertainer, and really really funny, but he's consistently wrong on AI stuff. He's got a BA (with honours I guess?) in media something or other. And again, he's really funny, but does that make him qualified to pontificate on matters of AI policy or economics or even to make statements about the technology etc?

Regardless, the misanthropy, the cynicism, and the misinformation that's being shouted about is because being a cynic is more "profitable" than being an optimist when you measure your success more by how many people agree with you rather than actually being right and being able to do things.