The craziest thing about AI is you can just try it yourself and check if the claims are true.

I use Claude code and codex daily. They have become an integral part of my workflow.

There is no task that takes me a day that they can complete in five minutes.

Even with the lightning fast progress being made, it looks like LLMs are a decade or more away from being that good.

If AI can do your job for you, you should be the first to know. Just try it and see!

There are definitely tasks you can prompt an AI in 5 minutes that would take a whole day to do. One example is adding something to a CI pipeline and getting it to green (i.e. maybe you're adding your first ever e2e test), especially when your CI pipeline is painfully slow. e.g. if your pipeline takes 30 minutes to finish, and it takes around 10 tries to figure out all the random problems, that was easily a full day task before AI. Now I prompt AI to figure it out, which takes 5 minutes of active attention, and it figures it out for the rest of the day while I do other stuff.

Fundamentally it cannot be much better than how well we can write the spec and then validate the results.

It’s always gonna be a multi shot process. And it can already write code good enough. That’s no longer the bottleneck.

Further, Qwen 27b is such an incredible masterpiece for coding and it can run on consumer hardware today. Anthropic/OpenAI are gonna give up on coding models very soon. There’s not gonna be any money in it when you can run your own local model for significantly cheaper.

Qwen27b is not SOTA but the value is insane. You can basically use it for small tasks and then route harder problems to opus or sonnet and boom you’ve said a lot of money.

Not my experience. AI takes a lot less time doing tasks than myself. My current issue is that 2 out of 3 they don't produce the code that I want, so I either have to reprompt or do it myself. And the solution is simple: just accept their way; I'm just not there yet.

In any case, on that one time that AI works perfectly, it saves me hours of coding. So the potential is there...

Super trivial to hand verify 350kloc changes for sure.

For select tasks the latest LLMs can speed things up by an order of magnitude.

Best example I’ve found: translating code from one language to another where there’s a large corpus of existing acceptance tests.

Yep. It depends so much on task, expectations, ability to express what you want and whether the problem has been solved elsewhere or not.

The results are always so ridiculously different.

When you do any meaningful work, that is, not "generate a website with a fancy UI", you very much realize that AI can not, in fact, "do the work". They constantly make mistakes, and you have to spend about as much time writing the spec and checking the code as you'd have written the code

So the effect is just merely some kind of acceleration of "boilerplate code writing", which is very impressive for beginner coders who are mostly doing automateable, trivial tasks, but much less so once you start doing real concurrency / threading / embedded / etc work

> There is no task that takes me a day that they can complete in five minutes.

Five minutes is pushing it, but 15 minutes? Absolutely.

The delta isn't a day to 5 minutes, but a day to a half hour (where most of my larger tickets take)? Yes, especially as you don't need to watch it do its thing anymore.

To me, the lack of amazing productivity gains is that we have done nothing to speed up figuring out what to build and nothing to speed up getting code into production from pull request and in a lot of companies, code review is already saturated.

Also, the agents are good at figuring out problems for themselves, so I can ask it to set up a CI/CD pipeline, give it GitHub access, and it will just try things until it succeeds.

Agreed with you. Non argumentative, just want to add to the convo: what's even more crazy is the cognitive dissonance around this idea.

> There is no task that takes me a day that they can complete in five minutes.

It's highly dependent on task. I was watching a podcast with Simon Wilson, where he said something like, (paraphrasing) "My whole selling point as a dev was that I could ship POCs / MVPs fast. Now that's somewhat obsolete."

It resonated with me because I feel like that also was a skill that I cultivated and excelled at. I agree with Simon's general thesis: that skill is largely dead. There are many pedants and detractors that will race to the defense of this art with various arguments to try to challenge the idea, but they simply do not hold up to reality. I have non-programmer friends with 10 dollar claude code subscriptions whipping up products to solve niche problems in their life / job.

I offered to help one of my friends who's working on generating math exams based on curricula and seed problem sets. I taught him how to use git, he pushed the repo, I looked at the repo and it wasn't clear he needed me. Everything I could do would be related to scale / reliability / optimization. They don't need any of that, they just need to prompt the ai to say, "go burn some subscription tokens for my AP Calc track this year." There's a whole saas and c2c industry built around this problem that this guy just solved for 10 bucks a month.

Of course, there's much more depth to engineering then just cranking out prototypes. There is still "real engineering" to be done, and software will likely start focusing more towards specification / verification.

But a lot of the industry was built around the idea of speed of delivery / time to market to explore product fit and rapidly iterate. IMO frontier llms (private and open weights) have this largely solved. I can build and test ideas that would have taken me a weekend last year in half a day now, the majority of that time I can be talking to the llm via matrix while I'm out in the world.

[dead]