I can't imagine SWEs will be reduced to SDETs anymore than attorneys will be reduced to spell-checkers on AI powered case briefs.

I am a very AI-forward person, but hallucinations are becoming more pernicious than ever even as they get less frequent, especially if the code actually works. A human absolutely has to guide these processes at a macro level for sustainability for SaaS as it evolves with business needs.

Maybe for one and done systems with no maintenance/no updates/no security patches you can reduce humans to SDETs, but systems like that are more the exception than the norm.

IMO the code-generation for boilerplate and the improvement of copypasta quality are much bigger improvements than that.

PMs turning their brain off and letting the LLMs extrapolate from quick and dirty bashing of text into a template (or, PMs throwing customer feedback at a slackbot to generate a jira ticket form it) can be better than PMs doing nothing but passing ill-defined reqs directly into the ticket, but that's a low bar. And it doesn't by itself solve the problems of the details that got generated for this ticket subtly conflicting with the details that got generated for (and implemented) in a different ticket 8 months ago.

> Honestly, with the first step, it seems the PMs are already halfway there to implementation of the feature so I wonder if in the future they'll just do everything themselves

I'm guessing they've tried (or been induced to try by upper management), but given up because they don't know how to debug any problems that arise due to the LLM working itself into a corner.

Coding-agent LLMs act a lot like junior devs. And junior devs are: eager to write code before gathering requirements; often reaching for dumb brute-force solutions that require more work from them and are more error-prone, rather than embracing laziness/automation; getting confused and then "spinning their wheels" trying things that clearly won't work instead of asking for help; not recognizing when they've created an X-Y problem, and have then solved for their Y but not actually solved for the original problem X; etc.

The way you compensate for those inexperience-driven flaws in junior devs' approach, is to have them paired with, or fast-iteration-code-reviewed by, senior devs.

Insofar as a PM has development experience, it's usually only to the level of being a "junior dev" themselves. But to compensate for LLMs-as-junior-devs, they really need senior-dev levels of experience.

The good PMs know all of this, and so they're generally wary to take responsibility for driving the actual coding-agent development process on all but the most trivial change requests. A large part of a PM's job is understanding task assignment / delegation based on comparative advantage; and from their perspective, it's obvious that wielding LLMs in solution-space (as opposed to problem-space, as they do) is something still best left to the engineers trained to navigate solution-space.

lol

Just lol. Is this what you guys mean by productivity boost?

Comical. LLM’s aren’t all that great - it’s more that most orgs are horribly inefficient. Like it’s amazing how bad they are.

That’s why Elon succeeded with spacex - he saw how horrible inefficient the industry was. And used that thinking to take a gamble and it’s paid off.

> Then devs can copy paste this Jira ticket content into the LLM agent of choice

Super glad to have gotten out when I did...

If you do that someone still needs to make sure the details make sense which, from experience, sometimes they will and sometimes they won't. When I open tickets using automation I often back into the ticket from a running implementation that passes tests so the description is at least internally consistent but there are often still issues that need corrected.

Maybe for some subset of sotware (like CRM panels or something) PMs will do everything. But if you're projecting the way one sort of software (ie user-facing, business use oriented software) is developed and put to use with software writ large, then no I don't think so

Except... no one validates the generated tickets, and it's full of inaccuracies.

And then someone copy pastes it into Claude and now those inaccuracies become part of the code and tests.

I literally can’t tell if this comment is a joke or not.

> Honestly, with the first step, it seems the PMs are already halfway there to implementation of the feature so I wonder if in the future they'll just do everything themselves

Yes please, I've seen the vibecoded slop PMs put out every day because software engineering is simply not a skill they have, and I'd love to make a LOT of money fixing their crap once it dies in production <3

It's interesting how predictable some of this is.

The LLMs turn out fully formed clones of stuff for which there exists copious amounts of code openly searchable on the web doing the exact same thing.

LLMs require developer-like specification, task/subtask breakdown and detail where such example code already exists.

As a professional prior to LLMs, how many problems that you work on have many existing free solutions but you neglected to use that code and decided to spend days doing it yourself?

"We've invented the silver bullet from the book 'No Silver Bullets'"

Juxt's Allium https://juxt.github.io/allium/ is an interesting entry in this 'pseudo DSL' space to define and store system specifications and requirements. I think it's likely that this sort of 'persistent specifications to help bots work correctly' will be a good approach when things finally cool down a bit.

That's the kind of stuff where you would write a few lines of shell script or perl and not bother with the whole GTK stuff. Because GTK would be accidental complexity to the task (unless you used something like zenity).

This is one of the reasons I like the OpenBSD and suckless projects. There are solutions that are technically correct, but are overengineered.

That's easily more spec than script.

> You're completely right

I mean, no comment

ChatGPT 5.5 responds:

> What data should I retrieve, and where should I get it from? Please specify at least: ...

And it then goes on to ask just exactly what is necessary, being all constructive about it.

When the cycles are short enough, though, that is to some degree the right thing. That is, it's the right thing for things the users can then immediately see and give feedback on, because it lets them give feedback on something tangible.

It's the wrong thing for important things under the hood (like durability and security requirements) that are not tangible to them.

Just as poorly designed code can still compile. This is operator error, not a failure of the technology.

IME you give it very precise specifications and it still fucks it up.

When we talk about "the" bottleneck being specs it just isnt the case that it's the only thing LLMs do poorly. Theyre really bad at a lot of stuff in the SDLC.

They're also good at providing results which are bad but look ok if you either dont look too closely or dont know what you're looking for.

> The process didn’t work before because the person writing the requirements either put out vague requirements or bad requirements because they didn’t understand the business intent (or were careless).

You make it sound like writing good requirements is easy.

If it were easy we wouldn't need all these concepts around PMF, product pivots and the like. And even before that was Peter Naur's paper "Programming as Theory Building" [1].

If you truly understand the problem you're solving with software then requirements can be easy. But usually we don't, not right away, and so we have to build up our understanding of the problem first in order to solve it.

Even then, the problem we solve may not have been the problem paying users will have, so you can have "good requirements" and still have a bad business, or even the opposite where you somehow build a working business despite bad requirements, because you hit upon a customer's need quite by mistake.

Nothing about any of this precludes LLMs being helpful, though nothing guarantees LLMs will be helpful either.

[1]: https://cekrem.github.io/posts/programming-as-theory-buildin...

Plausible requirement generators as inputs to plausible code generators.. what could go wrong!

Can someone pull that Steve jobs quote out re. The craftsmanship between a great idea and great product?

Anyone who thought that gap could be shrunk substantially lives in delululand.

Hence why we haven’t seen this explosion of ‘really great’ products come out.

Many will continue to parrot ‘bro but the models changed I swear’. I’m sure they did. But you’re missing the damn point.

But horribly token inefficient.

Did everyone forget about outsourcing and how outsourcing works?

The dudes in Eastern-Wherever not asking what something means is the scary part. You only find out at the end how deeply confused everyone was when making the thing. You can fix it with attention and management, but then only some projects sometimes are profitably outsourced and you still need competency.

Do they have a point?

Can't good marketing teams, backed up by World Class Product people, sell anything we build, more or less?

</devil's advocate>

The solution I've seen work is have engineers and designers that can take much of the detailed spec writing on, and have the PMs spend time with users/prospective users, partners, etc, understanding the market and users better. When you pull PMs in to all the details, often they turn into project managers, shuffling bug tickets around etc, taking time away from owning the user and the problem and shifting them too much to the solution side. Have a lead engineer own much / most of that. Every org / product is different of course.

I’ve never understood engineers who complain about vague specs… if the spec was complete, it would be code and the job would be done already! Getting a 20 page spec delivered from upon high and mechanically translating it to code without any chance to send feedback up the chain sounds like… a compiler.