True but the n00b is very fast. A lot of coaching is waiting for the n00b to perform tasks and meta things about motivation. These LLM are extremely fast and eager to work.

I don't need a study to tell me that five projects that have been stuck in slow plodding along waiting for me to ever have time or resources for nearly ten years. But these are now nearing completion after only two months of picking up Claude Code. And with high-quality implementations that were feverdreams.

My background is academic science not professional programming though and the output quality and speed of Claude Code is vastly better than what grad students generate. But you don't trust grad student code either. The major difference here is that suggestions for improvement loop in minutes rather than weeks or months. Claude will get the science wrong, but so do grad students.

(But sure technically they are not finished yet ... but yeah)

It is in many ways much like coaching a n00b, but a n00b that can do 10 hours of n00b work in 10 minutes (or, 2 minutes).

That's a significant difference. There are a lot of tasks that can be done by a n00b with some advice, especially when you can say "copy the pattern when I did this same basic thing here and here".

And there are a lot of things a n00b, or an LLM, can't do.

The study you reference was real, and I am not surprised — because accurately gauging the productivity win, or loss, obtained by using LLMs in real production coding workflows is also not junior stuff.

"Really does not sound like that from your description. It sounds like coaching a noob, which is a lot of work in itself."

And if this is true, you will have to coach AI each time whereas a person should advance over time.

> Really does not sound like that from your description. It sounds like coaching a noob, which is a lot of work in itself.

Even if you do it by yourself, you need to do the same thinking and iterative process by yourself. You just get the code almost instantly and mostly correctly, if you are good at defining the initial specification.

> Wasn’t there a study that said that using LLMs makes people feel more productive while they actually are not?

On a tangent; that study is brought up a lot. There are some issues with it, but I agree with the main takeaway to be weary of the feeling of productivity vs actual productivity.

But most of the time its brought up by AI skeptics, that conveniently gloss over the fact it's about averages.

Which, while organizationally interesting, is far less interesting than to discover what is and isn't currently possible at the tail end by the most skillful users.

Anecdotally, on green field projects where you are exploring a new domain - it’s an insanely productive experience. On mundane day to day tasks it probably takes more time, but feels like less mental bandwidth.

Coding at full throttle is a very intensive task that requires deep focus. There are many days that I simply don’t have that in me.

Sure it’s a lot of work, but the noob in question has all the internet knowledge and can write multiples times faster than a human for a fraction of the costs. This is not about an individual being more productive, this is about business costs. Long term we should still hire and train juniors obviously, but short term there is lot of pressure to not do it as it makes no sense financially. Study or not the reality is there is not much difference in productivity between a senior with a cursor license and a senior and a junior that needs heavy guidance.

LLMs make two people more productive, the person that uses the LLM, and then the person that cleans up the mess.

There was one study that said that in a specific setting and was amplified heavily on forums by anti-AI people.

There have been many more studies showing productivity gains across a variety of tasks that preceded that one.

That study wasn't necessarily wrong about the specific methodology they had for onboarding people to use AI. But if I remember correctly it was funded by an organization that was slightly skeptical of AI.

Everyone using that study to prove LLMs are bad hasn't actually read the study.

My buggy executive function frequently gets in the way of putting code to screen. You know how hacker news has that lil timeout setting to pseudo force you to disengage from it? AI made it so I don't need anything like that. It is digital Adderall.

You aren't wrong in the coaching but, but feedback loops are orders of magnitude faster.

It takes an LLM 2-20 minutes to give me the next stage of output not 1-2 days (week?). As a result, I have higher context the entire time so my side of the iteration is maybe 10x faster too.

I am so tired of this style of "don't believe your lying eyes" conjecture.

I'm a career coder and I used LLMs primarily to rapidly produce code for domains that I don't have deep experience in. Instead of spending days or weeks getting up to speed on an SDK I might need once, I have a pair programmer that doesn't check their phone or need to pick up their kids at 4:30pm.

If you don't want to use LLMs, nobody is forcing you. Burning energy trying to convince people to whom the benefits of LLMs are self-evident many times over that they are imagining things is insulting the intelligence of everyone in the conversation.

It's this, but 1000 times faster — that's the difference. It's sending a noob away to follow your exact instructions and getting results back in 10 seconds instead of 10 hours.

I don't have to worry about managing the noob's emotions or their availability, I can tell the LLM to try 3 different approaches and it only takes a few minutes... I can get mad at it and say "fuck it I'll do this part myself", the LLM doesn't have to be reminded of our workflow or formatting (I just tell the LLM once)

I can tell it that I see a code smell and it will usually have an idea of what I'm talking about and attempt to correct, little explanation needed

The LLM can also: do tons of research in a short amount of time, traverse the codebase and answer questions for me, etc

it's a noob savant

It's no replacement for a competent person, but it's a very useful assistant

I don't think it replaces a junior, but it raises the bar for the potential that a a junior would need to show early, for exactly the reason you mention. A junior will now need to be a potential senior.

The juniors that are in trouble are the low-potential workhorse folks who really aren't motivated but happened to get skilled up in a workshop or technical school. They hopped on the coding wagon as a lucrative career change, not because they loved it.

Those folks are in trouble and should move on to the next trend... which ironically is probably saying you can wrangle AI.

> work mentoring the AI strikes me as less valuable then the same time spent mentoring a junior developer

But where can you just "mentor" a junior? Hiring people is not so easy, especially not ones that are worth mentoring. Not every junior will be a willing, good recipient of mentoring, and that's if you manage to get one, given budget constraints and long lead times on hiring. And at best you end up with one or two; with parallel LLMs, you can have almost entire teams of people working for you.

I'm not arguing for replacing juniors - I worry about the same thing you do - but I can see why companies are so eager to use AI, especially smaller startups that don't have the budgets and manpower to hire people.

The junior could use the LLM as a crutch to learn what to learn. Whatever output the LLM gave them, they could examine or ask the LLM to explain. Don't put into production anything you don't understand.

Though I'm extremely well versed in Python, I'm right now writing a Python Qt application with Claude. Every single Qt function or object that I use, I read the documentation for.

I would spend time mentoring a junior, but I don't have one so I work with AI. It was the company's decision, but when they asked me "who can continue developing and supporting system X" the answer is "the nobody that you provided". When you cut corners on growing juniors, you reap what you sow.

It's a classic short-term gain outlook for these companies.

Yep, and the productivity of LLMs means that experienced developers can go from idea to implementation way faster. But first someone has to understand and define a solid structure. And later someone needs to review, test, and integrate the code into this framework. This is hard stuff. Arguably harder than writing code in the first place!

It's no wonder inexperienced developers don't get as much out of it. They define a vague structure, full of problems, but the sycophantic AI will spew out conformant code anyways. Garbage in, garbage out. Bad ideas + fast code gen is just not very productive in the long term - LLMs have made the quality of ideas matter again!

I think the idea was that LLMs can allow someone who has no idea how to code, to write a prompt that can in fact output some working code. This is greatly raising their skill floor, as opposed to a senior where at best it’s doing something they already can do, just faster.

The elephant in the room being that if you aren’t senior enough to have written the code you’ll probably run into a catastrophic bug that you are incapable of fixing (or prompting the LLM to fix) very very quickly.

Really it’s just the next iteration of no-code hype where people dream of building apps without code, but then reality always come back to the fact that the essential skill of programmers is to understand and design highly structured and rigid logical systems. Code is just a means of specification. LLMs make it easier to leverage code patterns that have been written over and over by the hundreds of thousands of programmers that have contributed to its training corpus, but they can not replace the precision of thought needed to make a hand-wavy idea into a concrete system that actually behaves in a way that humans find useful.

I've never worked anywhere where the role of a Sr was to glue together a bunch of small pieces written by a team of Jr devs.

I've only worked places where Jr's were given roughly the same scope of work as a mid-level dev but on non-critical projects where they could take as much time as necessary and where mistakes would have a very small blast radius.

That type of Jr work has not been made redundant - although I suppose now its possible for a PM or designer to do that work instead (but if your PMs are providing more value by vibe coding non-critical features than by doing their PM work maybe you don't really need a PM?)

I don't see this a problem of seniority but one of mindset. I've met enough "senior devs" that will push just about anything and curious juniors that are much more selective about their working process.

IMHO, not really, if you know what you want.

There will always be small things to fix, but if there needs to be a second draft, I would hazard that the PR was too big all along: a problem whether an AI is involved or not.

My last attempt had passing tests. It even exercised the code that it wrote! But, upon careful inspection, the test assertions were essentially checking that true equalled true, and the errors in the code didn't fail the tests at all.

Attempting to guide it to fixing the errors just introduced novel errors that it didn't make the first time around.

This is not what I signed up for.

Byzantine Incompleteness enters the chat.

Either you go formal, or you test the tests, and then test those ...

For me I only use it as a second opinion, I got a pretty good idea of what I want and how to do it, and I can ask any input on what I have written. This gives me the best results sofar.

Have you tried a more granular strategy - smaller chunks and more iterative cycles?

This only happens if you want it to one-shot stuff, or if you fall under the false belief that "it is so close, we just need to correct these three things!".

Yes I have encountered it. Narrowing focus and putting constraints and guiding it closer made the LLM agent much better at producing what I need.

It boils down to me not writing the code really. Using LLMs actually sharpened my architectural and software design skills. Made me think harder and deeper at an earlier stage.

I am not the biggest fan of LLMs but I have to admit that, as long as you understand what the technology is and how it works, it is a very powerful tool.

I think the mixed reports on utility have a lot to do with the very different ways the tool is used and how much 'magic' the end-user expects versus how much the end-user expects to guide the tool to do the work.

To get the best out of it, you do have to provide significant amount of scaffolding (though it can help with that too). If you're just pointing it at a codebase and expecting it to figure it out, you're going to have mixed results at best. If you guide it well, it can save a significant amount of manual effort and time.

I can imagine it often being the case that if you measure a concise moderately difficult task over half a day or a few days, coding by hand might be faster.

But I think, and this is just conjecture, that if you measure over a longer timespan, the ai assisted route will be consistently faster.

And for me, this is down to momentum and stamina. Paired with the ai, I’m much more forward looking, always anticipating the next architectural challenge and filling in upcoming knowledge and resource gaps. Without the ai, I would be expending much more energy on managing people and writing code myself. I would be much more stop-and-start as I pause, take stock, deal with human and team issues, and rebuild my capacity for difficult abstract thinking.

Paired with a good ai agent and if I consistently avoid the well known pitfalls of said agent, development feels like it has the pace of cross country skiing, a long pleasant steady and satisfying burn.

The truth of it is that when I code with an LLM I scope the work up to include parts that would be a stretch for me to implement. I know what I want them to do, I know where I could find the info to write the code, but the LLM can just spit it out and if it's validate-able, then great, on to the next.

If I were to attack the same system myself without any LLM assist, I'd make a lot of choices to optimize for my speed and knowledge base. The code would end up much simpler. For something that would be handed off to another person (including future me) that can be a win. But if the system is self contained then going bigger and fancier in that moment can be a win. It all depends on the exact goals.

All in all, there's a lot of nuance to this stuff and it's probably not really replacing anyone except people who A) aren't that skilled to start with and B) spend more time yelling about how bad AI is than actually digging in and trying stuff.

> the AI writes the coding route is slower, though the developer percieves it as faster.

I have this pattern while driving.

Using the main roads, when there is little to no traffic, the commute is objectively, measurably the fastest.

However, during peak hours, I find myself in traffic jams, so I divert to squiggly country roads which are both slower and longer, but at least I’m moving all the time.

The thing is, when I did have to take the main road during the peak traffic, the difference between it and squiggly country roads was like two to three minutes at worst, and not half an hour like I was afraid it would be. Sure, ten minutes crawling or standing felt like an hour.

Maybe coding with LLMs makes you think you are doing something productive the whole time, but the actual output is little different from the old way? But hey, at least it’s not like you’re twiddling your thumbs for hours, and the bossware measuring your productivity by your keyboard and mouse activity is happy!

Even if that's true today ( it's not ), it becomes less true over time as tools and models improve.

If you have someone who knows what they're doing with the latest and greatest coding agents it's just not comparable. You can have a Dev open up four or more terminals with multiple prompts running at the same time. A manual person just can't keep up.

Once you get a sense for LLM workflow, sometimes the task is not appropriate for it and you do write by hand. In fact, most code I write is by hand.

But if I want a new system and the specs are clear, it can be built up in stages that are testable, and there are bits that would take some research but are well documented… then it can be a win.

The studies that say devs are slower with LLMs is fair because on average, devs don’t know how to optimize for them. Some do though.

The massive productivity gains I’ve seen come from multidisciplinary approaches, where you’d be applying science and engineering from fields like chemistry, physics, thermodynamics, fluids, etc, to speedy compiled languages. The output is immediately verifiable with a bit of trial and error and visualization and you’re saved literally months of up front text book and white paper research to start prototyping anything