"We believe our internal AI R&D efforts are significantly faster than they would be without AI assistance, but not yet by a factor of 2 (though we are uncertain and measurement is difficult)"

So Anthropic thinks their productivity is not even doubled by AI. Interesting data point.

▲

furyofantares an hour ago | parent | next [-]

> So Anthropic thinks their productivity is not even doubled by AI.

I find it hard to imagine launching this criticism at a new technology.

▲

camdenreslink an hour ago | parent | next [-]

Some people (not Anthropic necessarily) are claiming 5x or 10x productivity improvement. So it is relevant, as it is being somewhat oversold by some.

	▲	Frost1x 43 minutes ago \| parent \| next [-]
		To play the devil’s advocate, some people and orgs that were highly inefficient that adopted it really could get massive productivity gains. A performance improvement is relative to some baseline, and that baseline for some may be a lot lower than others, and if they adopt tech effectively it really could be a big boost. Across the board though I don’t think it’s sensible. I work in an industry that very intentionally tries to be inefficient and I can tell you having certain tasks automated that before had a person barrier intentionally acting inefficiently that you can now sidestep by outsourcing their tasks to something like Claude gives me a massive performance increase because I’m not blocked as much anymore. I can literally just replace some external tasks that were intentionally slowing processes down for their own benefits with a few prompts and move along. I could have done the tasks before but then people would ask why I’m spending my time doing it, now I can just say “oh, I was blocked so I had Claude take care of that blocker” and move along.
	▲	furyofantares 32 minutes ago \| parent \| prev \| next [-]
		It can be true that some people are getting 5x or 10x while Anthropic is getting less than 2x. CRUD apps that are simple variations on something done a zillion times before is different than AI research. R&D is not some thing that just requires someone to type a volume of code. It can further be true that some developers are getting 5x or 10x while as a whole their organization is sped up less than 2x. I'm sure many tasks at Anthropic are sped up 5x or 10x or more. It can further be the case that many people overestimate their gains as well. That's fine, and I think what you're saying - but it's still wild to shake your head and go "pfft, they have not even doubled their productivity". Double is a lot!
	▲	ogig an hour ago \| parent \| prev [-]
		Not all task are cutting-edge AI research.

▲

jchw an hour ago | parent | prev [-]

Now let's re-evaluate that based on how much it costs in both R&D and at runtime. This new technology has a lot of work to do to justify itself.

	▲	s1artibartfast an hour ago \| parent [-]
		Doesn't seem crazy to me. Anthropic investment has been about 80 billion cash and 50 billion in services that mostly went back to the investors. Seems like that would be easily recupable even without future growth. While it is true that investors only got a fraction of the company for that money, and their EV can be debated, I think the clear the value is there from a net cash in to value produced. Last valuation was like 1 trillion. Company could be worth like 1/20 and still justify the cash.

▲

dangets 12 minutes ago | parent | prev | next [-]

It is also difficult to measure this in the face of ever shifting baselines. Anthropic is built on AI from the ground up. I'm not picking a side, but Anthropic getting a 2x increase is different bar than some random enterprise shipping legacy apps with clunky processes getting a 2x increase by adopting.

▲

bonoboTP an hour ago | parent | prev | next [-]

Well, it is a data point but AI R&D at a frontier lab is not really a representative stand-in for a regular workplace.

▲

T0Bi an hour ago | parent | prev | next [-]

AI R&D efforts != productivity. I think it's fairly obvious that SOTA research is less affected by AI than writing another boilerplate react frontend.

▲

impossiblefork an hour ago | parent | prev | next [-]

I think LLMs are best for ideation, experiments, small things.

They've probably already settled on most of the architecture and the big ideas, so they're details in big things instead of how to make complete small things.

The thing LLMs really speed up is how some ordinary person-- a PhD student, or similar, can whip up a miniature synthetic experiment that turns out to be horrid and needs to be fixed by hand, but which at least gave him a plot on the same day he had the idea. That's, I think, where LLMs shine: prototypes. Anthropic probably doesn't need that to the same degree as the small experimenter.

▲

an hour ago | parent | prev | next [-]

[deleted]

▲

OJFord an hour ago | parent | prev | next [-]

R&D productivity. Pretty sure they claim more for e.g. Claude Code.

▲

what an hour ago | parent | prev | next [-]

>thinks

They can’t measure even measure it, it’s just vibes. They may not even be more productive.

	▲	scj an hour ago \| parent [-]
		To be fair, there isn't a good method of measuring software development productivity in general. Maybe they should ask an AI to create one!

▲

whateveracct an hour ago | parent | prev | next [-]

So they did two years of work in one year? yeah right lol

	▲	AlexB138 an hour ago \| parent \| next [-]
		This is the fastest growing software company in history. They probably did more work in the last year than most companies do in their entire existence.
	▲	semiquaver an hour ago \| parent \| prev [-]
		I mean, look at what they’ve released in the last year. I think most companies would be proud to have that done in three. Say what you will about Anthropic but they ship.

▲

taosx an hour ago | parent | prev [-]

So I can take that as ~infinite amount of tokens don't don't even get you 2x on any novel tasks? No auto-researcher, no rsi..

Is that correct?