Remix.run Logo
bryan0 2 days ago

I agree. There are 2 groups of people:

1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.

2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.

The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.

mrandish 2 days ago | parent | next [-]

> in fact it does the opposite because of the need for government regulation.

For frontier leaders, regulation is a competitive moat. At trillion dollar scale, compliance teams, legal and lobbyists are a tiny fraction of opex. For emerging startups, early regulation of a new market is a substantial barrier to entry.

That said, I think Anthropic blundered into their current Fable mess unintentionally by underestimating just how much they pissed off the current administration by refusing to let the DoD use Ant's AI's for all the domestic spying and lethal combat ops they want.

ekelsen 2 days ago | parent | prev | next [-]

I don't think the first position is steel manned by assuming those holding it actually believe there are no potential problems or ways to use the technology badly.

But that belief does not imply the doom rhetoric is necessary or the best way to approach those issues.

shard972 2 days ago | parent [-]

[dead]

JMiao 2 days ago | parent | prev [-]

that's an inaccurate characterization on #1. he's not saying ai risk is _all_ hype to boost valuations, but that it's misused to serve purposes other than warning.

bryan0 2 days ago | parent [-]

Those words come directly from TFA:

> It’s all just nonsensical hype

And he’s referring to completely real and reasonable warnings in anthropic blog posts about the exponential rate of AI development.

zozbot234 2 days ago | parent | next [-]

What's "exponential" about AI development? Model parameter counts? Anthropic doesn't publish those for their own models, last I checked. Datacenter buildouts? Water consumption per request? There just isn't enough evidence that AI smarts is growing all that much, once you account for the scaling of inputs. That's what OP probably means by "nonsensical hype".

ben_w 2 days ago | parent [-]

What's "exponential" about AI development?

The METR task-completion time horizons, for one.

https://metr.org/time-horizons/

zozbot234 a day ago | parent [-]

Lousy benchmark, they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.

ben_w a day ago | parent [-]

> Lousy benchmark

Make your own then. It can go on the pile with all the others that keep getting saturated too fast to be useful.

> they explicitly focus on the easiest tasks to automate for AI (i.e. heavily cherry picked outcomes) and it seems that they don't bother to test anything except just-released proprietary models.

What?

They made the benchmark last year, and included a bunch of models going back as far as 2019.

When they first announced it, the top end of their tests were things AI could not actually automate, and even now only does erratically. Examples of the tasks SOTA models are now saturating (at the 50% success level, not at 80%) include:

  "Prune attention heads of a BERT language model while minimizing accuracy loss on text classification tasks."
  "Implement a Python library for the ACE-OAuth standard that can generate and parse messages in CBOR format and encrypt/decrypt access tokens with COSE according to RFC specifications."
  "Debug a PyTorch machine learning library with gradient calculation and memory optimization bugs until all tests pass."
  "Finetune a large language model to reduce the accuracy of a truth detection probe while maintaining performance on standard benchmarks."
 - https://arxiv.org/html/2503.17354v1
They're benchmarking against the time it takes humans to do the same things, which means everything they ask every AI to do must have also been done by a human.
suttontom 21 hours ago | parent | prev | next [-]

It's a blog. He's using hyperbole. It's not a Supreme Court opinion.

JMiao a day ago | parent | prev [-]

what the "tfa" doesn't say is that geohot writes like how most people talk. you're framing the observation you don't like around a single word and ignoring everything else.