BadThread2026 2 days ago

Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics. Here's an additional data point: I think polyamory is stupid, I've read the object-level arguments, and this entire thread is ridiculous.

Everyone who works at a frontier lab should be in prison, AND ALSO the people talking about 10% risk of human extinction are both sincere and correct despite their many personal failings, AND ALSO the frontier labs have already created AI that could Hugging Face a biolab with smallpox or worse, AND ALSO they aren't going to slow down or lose any customers if you adopt some galaxy brained strategy where you spitefully tell them they aren't allowed to stop accelerating because polyamory is stupid.

▲

huurtehoog 2 days ago | parent [-]

> Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics.

I think you might have meant hermeneutics, in the sociological sense of 'subjective interpretation'?

▲

BadThread2026 2 days ago | parent [-]

Possibly. That's not a word I trust myself to use correctly. What I meant is that the object-level claims require a lot of time and energy to evaluate, and it's sometimes reasonable to use a heuristic like "sex cults are not trustworthy" to save time, but if you've already read the arguments then the heuristic loses almost all value.

Yudkowsky and friends believe many things, and reasonable people can disagree about many of them, but "at least 10% risk of extinction if we don't slow down" is a relatively straightforward conclusion based on the offensive capabilities of unreleased models and the terrible state of biosecurity and cybersecurity.

Even without AI, the risk of extinction might be above 10% if people with the relevant skills were interested in using them that way. AI adds an alien willingness to achieve poorly specified goals at any cost.

	▲	huurtehoog 2 days ago \| parent [-]
		Hm I think in terms of "can I trust a group of people" we're talking about heuristics yes as in: "how can I quickly approximate an answer to the question of trust?"