Remix.run Logo
skybrian 2 hours ago

We all use the same Internet. I would find it strange for anyone who's seriously interested in AI to have never heard of Yudkowsky or Scott Alexander or the Rationalist movement. I'd expect anyone interested in the field to have at least tried reading some of their articles over the years.

Like, if you never heard of them, were you living in a cave or something? It would be like never having heard of Y Combinator or Richard Stallman.

On the face of it, this is no more suspicious than reading the same best-selling books. Being in touch with what's going on doesn't necessarily mean people agree.

throwaway13337 2 hours ago | parent | next [-]

That's true. But even saying I disagree with Yudkowsky and having an opinion about that /is/ a framing. My perspective has shifted because I read these things which makes me more homogeneous with the group even if I disagree.

It's the framing that is important. We fell into a particular framing. We don't have a vocabulary to think about this stuff in a different way. This is water to us. We don't see it.

I'd like to see some more entropy in the system. More out-of-distribution ways to think of things. Because I think we over-indexed on some pretty shitty ways to see the world.

MarkusQ an hour ago | parent | next [-]

> More out-of-distribution ways to think of things.

Openly admitting that it only caught fire once the Plagiarism-as-a-Service took hold is a good first step. It's the part that everyone knows and few will openly acknowledge.

kridsdale1 2 hours ago | parent | prev [-]

Talking to the incredible talent in the Chinese Labs is likely a good way to do that.

GeneralMayhem 2 hours ago | parent | prev | next [-]

There's a difference between, on the one hand, knowing who they are and having read a few chapters of HPMoR, which I agree is standard for anyone who spends enough time on the Internet, vs. on the other hand actually identifying as part of that culture. I know who Stallman is, I've even seen him speak in person, but I'm not a GNU zealot.

The people described in the article aren't just "people who know who Yudkowsky is", they're self-avowed EAs and active LessWrong members.

Gangway0829 6 minutes ago | parent | next [-]

You have to be quite unaware of your bubble if you think knowing about a niche Harry Potter fanfic is "standard" for people on the Internet.

riversflow 34 minutes ago | parent | prev [-]

maybe its standard for someone who ends up on hackernews, but I'm a 30 something and all of my friends are online people around my age and none of them have ever heard of the HPMoR. The internet is a huge place.

nradov 2 hours ago | parent | prev | next [-]

I've heard of them. There's no reason to expect that those self-promoting blowhards can predict the future more accurately than random chance. Those who take them seriously are engaging in hero worship and unscientific appeal to authority.

“Who is more foolish? The fool or the fool who follows him?” ― Obi Wan Kenobi

jeremyjh an hour ago | parent [-]

One question to ask yourself honestly - is what they were predicting 20 years ago that they were right about and you were wrong about? I’ve been following them that long without agreeing with them, but they’ve made better predictions than I have. If you know much about what happened this summer and haven’t updated at all on any of it… I guess you’ve found your religion too.

ambicapter 20 minutes ago | parent | next [-]

I love the assertion that they were right and that you're familiar with what they were right about, but never actually stating outright what they were right about. Super convincing.

afavour an hour ago | parent | prev | next [-]

I actually think that would be a valuable exercise. Were they right about everything 20 years ago? Or just some things? When did they say AGI would arrive? Should that perhaps influence the way we interpret their assumption that AGI is right around the corner?

jeremyjh 31 minutes ago | parent [-]

They were not right about everything and they - at least Yudkowsky - has not made specific predictions about "when AGI" will happen that I am aware of. But he did predict it is possible in our lifetimes, which I was very skeptical of. He did predict alignment would be hard - I was in the "why would we even hook it up to the internet?" crowd. He thought he should spend his time on that, and I wish he'd done more of that, or at least convinced others to do more of that.

stale2002 11 minutes ago | parent | prev | next [-]

> what they were predicting 20 years ago

Well one thing that they got wrong was how easy AI alignment ended up being in the end. I can fully understand have a basic view of alignment even 10 years ago, about how its difficult to explain human values to a computer.

But, looking around at whats out there today, it seems that computers are able to do a pretty good job of understanding human values without accidently believing its a good idea to turn the world into paperclips/computronium because you told it to make your computer run faster.

19 minutes ago | parent | prev [-]
[deleted]
BeetleB 2 hours ago | parent | prev | next [-]

> It would be like never having heard of Y Combinator or Richard Stallman.

I work in the tech sector. At one big company I worked for several years, 90% of the times I've mentioned Y Combinator to a colleague, I've had to explain who they are. And not one person I mentioned HN to had heard of it.

Likewise, no one knew levels.fyi. Not many knew Stallman, but why should they?

This isn't strange to me. That it's strange to you is evidence that you do live in a bubble and are likely susceptible to some groupthink. (I'm sure I am too, but for different bubbles from yours.)

I've read LessWrong stuff occasionally over the last decade. It rarely resonated with me. It's not bad, but it's not that great. I do tend to question someone's analytical skills when they go all in on that site. And I will say that even in the first few years of occasionally reading it, the group think was apparent.

rescbr an hour ago | parent | next [-]

Me too. I worked in two enterprise tech companies that definitely everybody here knows and I had the same experience with YC and HN. The two colleagues that knew about it, one went to found his own startup and went through YC, the other left the big co to work for a startup after about a year.

ryandrake an hour ago | parent [-]

Most people who work in computers/Internet do not live and breathe computer/Internet culture all day. They come in, do their job, clock out, and think about real life for the rest of their day. Most people I know, even people who work in tech, cannot name a single tech CEO other than their own company’s.

qlte an hour ago | parent | prev [-]

A while back I saw LessWrong recommended on HN in a list of similar communities that included lobsters.rs, etc. I thought the design of the site looked really nice so browsed with a open mind (I didn't make the rationalism/LW/Eliezer connection).

At the time, one of the recommended posts that was soft-stickied at the top for 1 week + was:

"How to Make Superbabies"

https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-mak...

Which, when I finally ended up clicking, was pretty much exactly what if sounds like from the title:

A long, meandering, manifesto full of selectively chosen charts and research findings with 38 mentions of "IQ", from a guy who has been unsuccessfully pitching investors on genetically modifying a grab bag of (dubiously) intelligence-associated genes in human babies.

Apart from the overall unhingedness, the part I found most striking while reading it was the off-the-chart level of self-confidence that flipping a bunch of genes on/off with CRISPR would have more or less zero risk of complications or unpredictable interactions. In his view, all that was holding back humanity's inevitable next step up the evolutionary ladder were the narrow-minded gatekeepers in the scientific establishment clinging to their so-called "ethics" and regressive "precautionary principle".

Without anything resembling experimentation to test his plan in animal models, yet alone early stage clinical trials for safety, he was willing to roll the dice on some unlucky baby dying in childbirth or living a life of agony from chronic disease based on some cherry picked papers he found on PubMed.

Because of the baffling hubris and my overall low impression of his intellectual honesty/rigor in the write up (clearly coming from a place of seeking to justify his own pre-existing beliefs), I opened the comments section expecting/hoping this was just some local nut who would be torn apart in the comment section.

But lo and behold, the very first comment at the top:

  Eliezer Yudkowsky

  One of the most important projects in the world.  Somebody should fund it.
... and thus ended my brief time trying out LW.
Gangway0829 2 minutes ago | parent [-]

My extremely uncharitable view of of these kinds of people, who now infest the American AI scene, is that they got cyber bullied by domain experts for saying the most ridiculous crap on repeat, and have therefore decided to spend their life on eliminating domain expertise.

kelnos an hour ago | parent | prev | next [-]

I think it's telling of your bubble that you think anyone who works in tech has heard of YC or Stallman.

As for Yudkowsky and Alexander, I've known about them for at least a decade, but I wouldn't be surprised if a lot of folks who are "seriously interested in AI" haven't heard of them.

rescbr an hour ago | parent | prev | next [-]

I just heard about them today. The only other person mentioned in the story that I've heard about once was Aella.

I mean, why should I have heard about them if I'm not in SV?

kridsdale1 2 hours ago | parent | prev | next [-]

I do use the internet. I did find their writings. I did read them. My calibrated bullshit detecting intellectual malware filters immediately flagged the content and it failed to imprint itself on my neural net in the next weights update.

timmytokyo 2 hours ago | parent | next [-]

Exactly. And companies like OpenAI and Anthropic are disproportionately populated by people deeply connected to Effective Altruism or the Bay Area rationalist movement.

bigyabai 2 hours ago | parent | prev [-]

Yeah. Just a few years ago, LessWrong was derided as the home of conceited, low-quality opinion pieces. Now there's a bizarre marketing push to launder their credibility and force them into the spotlight.

I imagine that thousands of talented AI researchers around the world have never read a line of Yudkowsky's work. They're probably not any worse-off for ignoring him, I'd say.

runmanny 30 minutes ago | parent | prev [-]

[dead]