The elephant in the room here is that the METR report itself was researched and compiled almost entirely by AI, with only very limited human "spot checks."

So I'm really not sure how much of it can be believed, especially since AI agents are strongly biased about the capabilities of AI agents.

▲

nater5000 10 minutes ago | parent | next [-]

I think there's two factors that are worth considering when it comes to this:

First, there's an element of timeliness that simply has hard constraints. In order to perform a "proper" analysis of this situation (i.e., little to no dependence on AI tools), you'd have to expect a pretty long wait. I know I'd rather have some sort of "initial report" as quickly as possible than to wait a year or two to get a report about a situation that will likely look trivial in a year or two. I imagine we'll see more detailed, human-developed reports over longer time ranges.

Second, I suspect the expectation of non-AI driven reporting of these kinds of things will definitely decline rapidly as everything scales up quickly. I mean, the data being produced by situations like this comes in the form of natural language "forum posts" (so to speak), but done at an autonomous scale. This isn't a collection of emails and Slack messages posted by humans in an org over the course of a few months; this is a bunch of bots interacting with each other in relatively novel ways as quickly as possible. It is, unfortunately, a perfect job for LLMs.

None of this disagrees with your points, necessarily. But I just think it's worth pointing out that this doesn't seem like a case of "And look! METR is so confident in LLMs that we're able to use it instead of paying humans to save a buck :D" and more of "Without LLMs, we'd only be half-way done analyzing this data before there are dozens more such investigations on the docket, so this will have to do."

▲

cubefox 6 minutes ago | parent | prev | next [-]

The main author talks about this problem here: https://www.lesswrong.com/posts/FG54euEAesRkSZuJN/ryan_green...

▲

Catloafdev an hour ago | parent | prev | next [-]

Do you have a source for this claim? Because the article literally starts by listing the real people who contributed to this.

Edit: I should have read through the whole thing first, ignore me

▲

lukev an hour ago | parent | next [-]

From the report:

> Because there were over a thousand transcripts and most were extremely long, we had to heavily delegate our analysis to AI agents; these agents had significantly worse judgment and reliability than human researchers, and it was challenging to spot check their work because both the underlying data and the agents’ analysis of it was often difficult to interpret.

> We estimate we spent roughly ~$400K in API credits over the six days of our investigation.

I don't understand why you think it's conceptually absurd? I use agents to analyze complex production issues all the time and they are very much capable of hallucinating a narrative.

	▲	Catloafdev an hour ago \| parent [-]
		I appreciate the response, I should have finished reading through the whole thing first. My initial reaction assumed far less usage of AI to analyze the data.

▲

alextheparrot an hour ago | parent | prev | next [-]

https://www.lesswrong.com/posts/FG54euEAesRkSZuJN/ryan_green...

▲

StevenWaterman 43 minutes ago | parent | prev [-]

TFA says as much, and METR said so themselves

▲

timmytokyo an hour ago | parent | prev [-]

One must also consider the well-known biases and motives of the authors. They are going to do everything they can to create hype around threats posed by AI.

METR is a cog in the effective altruism machine. It was spun off from Paul Christiano's Alignment Research Center. Christiano is a well-known longtermist and AI doomer, who predicts a 50% chance that AI will end humanity once it reaches human capacity [1].

The author of this piece is also a well-known member of the Bay Area rationalist cult.

[1] https://www.businessinsider.com/openai-researcher-ai-doom-50...