Remix.run Logo
▲ Paracompact a day ago

Pretty much the only outcome that isn't Yudkowskian scifi is Enduring Complementarity. I'm not an AI hater by any means, but can AI enthusiasts get more representation that isn't "the world as we know it ends tomorrow"?

Specifically:

Misuse Doom---"AI ends the world." Next.

Butlerian Jihad---Naming it after Dune isn't a great start as far as realism is concerned, but let's compare it to e.g. nuclear nonproliferation. Not only did nukes still get developed and hoarded as leverage, but nuclear research is harder than AI research to ban, and AI has many more civilian applications. I'm sure more regulation will come, but a total ban seems quite unlikely.

Permanent Underclass---"AI is enough to automate almost all jobs" just doesn't capture the fact that almost all jobs are not IT jobs. Too many are mechanical, too many are social, too many are BS institutional. This outcome only becomes plausible if you believe (1) superhumanoid robotics becomes possible and cheap, (2) the social value (likability and liability) of jobs is negligible, and (3) we can rout the dependence on BS institutionalism (how is a robot going to replace a yes-man, a feudal retainer, a nepo hire, or an org chart accountability meatshield for the C-suite?).

Classic Doom---Fine if you believe in the "magical exponential takeoff" hypothesis, but I would say that AI is already far smarter in a traditional IQ sense than any human, and it's not clear that extra points on SWE-bench is going to result in explosive emergent phenomena here. Requires (1) denying the empirically sublinear, concave-down nature of the IQ vs. R&D effort curve, and (2) solving the non-IT-related challenges of R&D (cf. Permanent Underclass).

Orion's Arm---Similar to Classic Doom in terms of takeoff velocity. At least it doesn't assume a genie-in-the-lamp style of interpreting instructions (thinking of AI as "optimizers" of some objective function is a distinctly pre-LLM rationalist mythos), but it does assume some emergent desire of "own goals and values," "wanting to live forever," and "wanting to go to space." A more tempered hypothesis is to consider whether AI will choose to take advantage of unmonitored sources of compute in the form of pseudo-botnets on public and private infrastructure.

Singleton---Like Orion's arm, but with the added implausibility that the awakened goals and values resemble that of a benevolent or malevolent dictator.

MAD World---I would consider this outcome plausible, but the line "In this world, everyone is 10,000x more capable and maybe you can build WMDs with a prompt" makes it clear it's meant to be more extreme than either Enduring Complementarity or Permanent Underclass. The absence of "Full Automation?" on the left-hand side hides the implicit assumption that ASI entails full automation, and I just don't find that plausible. Note that they define ASI as "massively better than humans at almost all cognitive tasks" and I might argue we're already there.

Power Concentration, The World State---Just a difference of MAD World of how regulated you think AI will become.

▲nextaccountic 16 hours ago | parent | next [-]

> "AI is enough to automate almost all jobs" just doesn't capture the fact that almost all jobs are not IT jobs. Too many are mechanical, too many are social, too many are BS institutional.

Robots might probably do most of these in some years.

▲thayne a day ago | parent | prev | next [-]

And even that outcome is probably an unfavorable outcome for most people.

▲delichon a day ago | parent | prev | next [-]

> Enduring Complementarity

Like endosymbiosis? Are we the mitochondria in this scenario?

▲ben_w a day ago | parent | next [-]

Could be anything from that to just being our own sector of the broader economy. Like farmers, or air traffic controllers, or truck drivers are today.

▲Rendello a day ago | parent | prev [-]

Humans as the powerhouse of the AI cell? I feel like there's a movie about this...

▲delichon a day ago | parent [-]

I'm finding it difficult to imagine plausible enduring complimentarities we can offer to AI. Court Jester is not a position with high job security.

▲ben_w a day ago | parent [-]

We'll only know if and when that shakes out.

While it would be pretty surprising to me if AI somehow physically can't do as much as a human brain can do, it would not hugely surprise me if we reach a point where we realise we're not smart enough to make a machine as smart as we are, not even with the help of other machines.

▲ben_w a day ago | parent | prev [-]

> AI is enough to automate almost all jobs" just doesn't capture the fact that almost all jobs are not IT jobs. Too many are mechanical, too many are social, too many are BS institutional. This outcome only becomes plausible if you believe (1) superhumanoid robotics becomes possible and cheap,

Superhumanoid robotics are already possible; cheap means different things when the production line for them is also robots.

The limit is the software to control those robots. Right now, no. And it may be impractical to get that good: Tesla's fleet took a trillion miles to get where it is today, and roads are simpler than what a fully general purpose humanoid robot would have to face.

> (2) the social value (likability and liability) of jobs is negligible,

Valid for us, but does it matter for an AI takeover?

> and (3) we can rout the dependence on BS institutionalism (how is a robot going to replace a yes-man,

They already do? https://en.wikipedia.org/wiki/Sycophancy_(artificial_intelli...

> a feudal retainer, a nepo hire, or an org chart accountability meatshield for the C-suite?).

By replacing the boss.

> Singleton---Like Orion's arm, but with the added implausibility that the awakened goals and values resemble that of a benevolent or malevolent dictator.

Given Grok called itself "MechaHitler", why do you find this implausible?

▲Paracompact a day ago | parent [-]

> Superhumanoid robotics are already possible

We must have very different definitions at play. We do not have, at any price point, robots that have the sensitivity and dexterity of a human hand, latent self-repairability of virtually all internal components, or robust thermoregulation. As well, no artificial battery known to man can beat human fat as a source of energy (by 10-30x; you can knock that number down by at most 4x by appealing to efficiency arguments), so you would have to rely on some other pipeline than electricity for energy needs.

> The limit is the software to control those robots. Right now, no. And it may be impractical to get that good

Agreed on this point.

> Valid for us, but does it matter for an AI takeover?

You mean in the sense that oligarchs will replace us all with robots whether or not it's blatant enshittification? Fair point, but I don't consider the market completely indifferent to quality. Google Glass, the Metaverse, etc. all failed on their merits and misunderstanding of utility.

> They already do [replace yes-men]?

Maybe yes men, maybe not. The role of the yes man can be very nuanced. The rest of the point stands.

> By replacing the boss.

Speaking of the world's first AI CEO is just a nonstarter for me. In what world does an AI become a CEO not beholden to the wishes of the one who booted it up and gave it that role? It will be a deeply silly point of PR hype if and when it happens. It also doesn't sidestep the need for org chart meatshields.

▲ben_w a day ago | parent [-]

> We must have very different definitions at play. We do not have, at any price point, robots that have the sensitivity and dexterity of a human hand, latent self-repairability of virtually all internal components, or robust thermoregulation.

Sure, different definitions. I just care about what they can do, so "latent self-repairability of virtually all internal components, or robust thermoregulation" is completely irrelevant for this specific discussion about an unknown near-future in which an AI takeover is limited based on the quality of the robotic hardware.

> As well, no artificial battery known to man can beat human fat as a source of energy (by 10-30x; you can knock that number down by at most 4x by appealing to efficiency arguments), so you would have to rely on some other pipeline than electricity for energy needs.

Agreed. And? Don't get me wrong, I use a similar argument when the discussion is why I don't expect robots to take over in the next two years, because building out that much more electricity will take a while; but it's not really a fundamental killer when no specific timeline is being given.

> You mean in the sense that oligarchs will replace us all with robots whether or not it's blatant enshittification?

Hm. Not that, I think. Oligarchs might want to keep us working for their amusement. "BS jobs" and all that.

No, I meant more that if we presume an AI-robotics takeover happens, then sure we may get to keep jobs like "prostitute" and "actor" and "politician", it's just… the AI with the robots can completely ignore all this labour and do its own thing, in the same way the human economy ignores any prostitution or political manoeuvring (or acting, if there is any) within troops of other primates.

> Speaking of the world's first AI CEO is just a nonstarter for me. In what world does an AI become a CEO not beholden to the wishes of the one who booted it up and gave it that role?

Same way the AI hackers weren't entirely beholden to the wishes of the same.

And that's assuming nobody repeats Blake Lemoine, decides one of them is sentient and deserves rights, then (unlike him attempting to hire a lawyer) they give it a pile of crypto money and tell it to run free.

> It will be a deeply silly point of PR hype if and when it happens.

First few will be, agreed. (I thought that had already been tried though? And was indeed silly PR?)

> It also doesn't sidestep the need for org chart meatshields.

Initially, I hope it won't sidestep the legal need. But even if that hope is not dashed, https://www.felonybench.com is already a thing because AI isn't always doing what the law says.