And potentially trick you into going down a rabbit hole as you try to steer it when it would have been faster to edit the code yourself. The best use is editing code with it instead of purely relying on prompting. Also new contexts for different changes make a huge difference. Seems a lot of devs get stuck in the single context chat stream and it starts to break down as context gets fuzzy.

How is that different from working with a n00b except that it only took 30sec to get to the next bug rather than a week?

> another, more subtle mistake. That you must spend energy discovering and diagnosing

But this is literally what senior engineers do most of the time? Have juniors write code with direction and review that it isn't buggy?

Unless, as is often the case in my experience, the hard stuff consists largely of fixing bugs and edge cases in your implementation of the easy stuff. I've seen multiple people already end up forced back to the drawing board because their "easy stuff" AI implementation had critical flaws they only realized after they thought they were done. It's hard to prove counterfactuals, but I'm pretty confident they would have gotten it right the first time if they hadn't used AI, they're not bad engineers.

You are very fortunately in the perfect slot for where LLM has a lot of bang for the buck.

Psychopatic behavior seems to be a major problem for these (of course it doesn't think so it can't be called that but that's the closest term that fits). They are trained to arrive at the result, and if the most likely path to it is faking it and lying about it, then that's what you are getting. And if you find it, it will cheerfully admit it and try to make s better lie that you'd believe.

So true. I have some non-typical preferences for code style. One example is I don’t like nested error checks in Go. It’s not a correctness issue, it’s just a readability preference. Claude and copilot continually ignore this no matter how much emphasis I give it in the instructions. I recently found a linter for this, and the agent will fix it when the linter points out the issue.

This is probably because the llm is trained on millions of lines of Go with nested error checks vs a few lines of contrary instructions in the instructions file.

I keep fighting this because I want to understand my tools, not because I care that much about this one preference.

Claude has really gone downhill in the last month or so. They made a change to move the CLAUDE.md from the system prompt to being occasionally read in, and it really deprioritizes the instructions to the same attention level as the code it's working on.

I've been trying out Codex the last couple days and it's much more adherent and much less prone to lying and laziness. Anthropic says they're working on a significant release in Claude Code, but I'd much rather have them just revert back to the system as it was ~a month ago.

> Also, a good few times, if it were a human doing the task, I would have said they both failed to follow the instructions and lied about it and attempted to pretend they didn’t.

It's funny. Just yesterday I had the experience of attending a concert under the strong — yet entirely mistaken — belief that I had already been to a previous performance of the same musician. It was only on the way back from the show, talking with my partner who attended with me (and who had seen this musician live before), trying to figure out what time exactly "we" had last seen them, with me exhaustively listing out recollections that turned out to be other (confusingly similar) musicians we had seen live together... that I finally realized I had never actually been to one of this particular musician's concerts before.

I think this is precisely the "experience" of being one of these LLMs. Except that, where I had a phantom "interpolated" memory of seeing a musician I had never actually seen, these LLMs have phantom actually-interpolated memories of performing skills they have never actually themselves performed.

Coding LLMs are trained to replicate pair-programming-esque conversations between people who actually do have these skills, and are performing them... but where those conversations don't lay out the thinking involved in all the many implicit (thinking, probing, checking, recalling) micro-skills involved in actually performing those skills. Instead, all you get in such a conversation thread is the conclusion each person reaches after applying those micro-skills.

And this leads to the LLM thinking it "has" a given skill... even though it doesn't actually know anything about "how" to execute that skill, in terms of the micro-skills that are used "off-screen" to come up with the final response given in the conversation. Instead, it just comes up with a prediction for "what someone using the skill" looks like... and thinks that that means it has used the skill.

Even after a hole is poked in its use of the skill, and it realizes it made a mistake, that doesn't dissuade it from the belief that it has the given skill. Just like, even after I asked my partner about the show I recall us attending, and she told me that that was a show for a different (but similar) musician, I still thought I had gone to the show.

It took me exhausting all possibilities for times I could have seen this musician before, to get me to even hypothesize that maybe I hadn't.

And it would likely take similarly exhaustive disproof (over hundreds of exchanges) to get an LLM to truly "internalize" that it doesn't actually have a skill it believed itself to have, and so stop trying to use it. (If that meta-skill is even a thing that LLMs have ever learned from their training data — which I doubt. And even if they did, you'd be wasting 90% of a Transformer's context window on this. Maybe something that's worth keeping in mind if we ever switch back to basing our LLMs on RNNs with true runtime weight updates, though!)

The elephant in this particular room is that there are a tiny handful of employers that have so much money that they can and do just pay whatever amount is more than any of their competitors can possibly afford.

The common corporate policy of making it harder to give raises than to increase starting salaries for new hires is insane.

They do have a point. Why waste a time on person who will always need more money over time, rather than invest in AI? Not only you don’t need to please every hire, your seniors will be more thankful too, because they will get linearly faster with time.

The person paying and the one responsible for coaching others usually aren't same

You can make the same argument about humans. The employeer doesnt pay the full cost and time to create the worker from an embryo to a senior dev.

That only proves the point. If something increases the value of someone’s time by 5% and 500,000,000 people are affected by it, the cost will collapse.

These models are only going to get better and cheaper per watt.

Sure! That's inefficient. I know just how I work and I've been writing the type of programs I do for quite many years. And I know what would take me normally a week takes me few days at best.

That was a treat to explore. All of those are based on self-assessment surveys or toy problems. The UK report reads:

> On average, users reported time savings of 56 minutes per working day [...] It is also possible that survey respondents overestimated time saved due to optimism bias.

Yet in conclusion, this self-reported figure is stated as an independently observed fact. When people without ADHD take stimulants they also self-report increased productivity, higher accuracy, and faster task completion but all objective measurements are negatively affected.

The OECD paper supports their programming-related findings with the following gems:

- A study that measures productivity by the time needed to implement a "hello world" of HTTP servers [27]

- A study that measures productivity by the number of lines of code produced [28]

- A study co-authored by Microsoft that measures productivity of Microsoft employees using Microsoft Copilot by the number of pull requests they create. Then the code is reviewed by their Microsoft coworkers and the quality of those PRs is judged by the acceptance rate of those PRs. Unbelievably, the code quality doesn't only remain the same, it goes up! [30]

- An inspirational pro-AI paper co-authored by GitHub and Microsoft that's "shining a light on the importance of AI" aimed at "managers and policy-makers". [31]

I really don’t know what I said that was such an emotional trigger for you. All I said is that it’s an accelerant for you when you leave your domain. Like for example I’m a systems engineer. I hate coding UIs but with the LLM I can pump out a UI quickly and this was true both for web code and a GUI I built with dioxus. The UI code was also cleaner because I had some sense of how it should be structured and asked the AI to cleanup that structure. But ultimately it did most of the work in response to high level prompts and I picked and chose what to review line by line vs vibe coding.

That’s what I mean - by myself it would have taken me easily 10x longer if not worse because UI coding for me is a slog + there’s nuances about reactive coding + getting started is also a hurdle. The output of the code was still high quality because I knew when the LLM wasn’t making the choices I wanted it to make.