I personally treat the LLM as a very junior programmer. He's willing to work, will take instructions, but his knowledge of the codebase, and patterns we use is lacking strongly. So it needs a LOT of handholding, very clear instructions, description of potential pitfalls, and smaller, scoped tasks, and reviewed carefully to catch any straying off pattern.

Also, I make it work the same way I do: I first come up with the data model until it "works" in my head, before writing any "code" to deal with it. Again, clear instructions.

Oh another thing, one of my "golden rule" is that it needs to keep a block comment at the top of the file to describe what's going on in that file. It acts as a second "prompt" when I restart a session.

It works pretty well, it doesn't appear as "magic" as the "make it so!" approach people think they can get away with, but it works for me.

But yes, I still also spend maybe 30% of the time cleaning up, renaming stuff and do more general rework of the code before it comes "presentable" but it still allows to work pretty quickly, a lot quicker than if I were to do it all by hand.

I think "junior programmer" (or "copilot") oversells the AI in some cases and undersells it in others. It does forget things that a regular person wouldn't, and it does very basic coding mistakes sometimes. At the same time it's better than me at some things (getting off-by-one errors when dealing with algorithms that work on arrays). It also has encyclopedic knowledge about basically anything out there on the internet. Red-black Trees? Sure thing. ECS systems for game programming? No problemo, here are the most used libraries.

I have ended up thinking about it as a "hunting dog". It can do some things better than me. It can get into tiny crevasses and bushes. It doesn't mind getting wet or dirty. It will smell the prey better than me.

But I should make the kill. And I should be leading the hunt, not the other way around.

The difference between LLM and a very junior programmer: junior programmer will learn and change, LLM won't change! The more instructions you put in the prompt, the more will be forgotten and the more it will bounce back to the "general world-wide average". And on next prompt you must start all over again... Not so with junior programmers ...

Definitely. To be honest I don't think LLM's are any different from googling and copying code off the Internet. Its still up to the developer to take the code, go over it, make sure its doing what its supposed to be doing (and only that) etc.

As for the last part, I've recently been getting close to 50 and my eyes aren't what they used to be. In order to fight off eye-strain I now have to tightly ration whatever I do into 20 minute blocks, before having to take appropriate breaks etc.

As a result of that time has become one of the biggest factors for me. An LLM can output code 1000x faster than a human, so if I can wrangle it somehow to do whatever basics for me then its a huge bonus. At the moment I'm busy generating appropriate struct of arrays for SIMD from input AoS structs, and I'm using Unity C# with LINQ to output the text (I need it to be editable by anyone, so I didn't want to go down the Roslyn or T4 route).

The queries are relatively simple, take the list of data elements and select the correct entries, then take whatever fields and construct strings with them. Even so, copying/editing them takes a lot longer than me telling GPT to select this, exclude that and make the string look like ABC.

I think there was a post yesterday about AI's as HUDs, and that makes a lot of sense to me. We don't need an all-powerful model that can write the whole program, what we need is a super-powered assistant that can write and refactor on a very small and local scale.

I personally see the LLM as a (considerably better) alternative to StackOverflow. I ask it questions, and it immediately has answers for my exact questions. Most often I then write my own code based on the answer. Sometimes I have the LLM generate functions that I can use in my code, but I always make sure to fully understand how it works before copy-pasting it into my codebase.

But sometimes I wonder if pushing a +400.000 lines PR to an open-source project in a programming language that I don't understand is more beneficial to my career than being honest and quality-driven. In the same way that YoE takes precedence over actual skill in hiring at most companies.

I find success basically limiting it to the literal coding but not the thinking - chop tasks down to specific, constrained changes; write detailed specs including what files should be changed, how I want it to write the code, specific examples of other places to emulate, and so on. Doesn’t have to be insanely granular but the more breadcrumbs the higher chance it’ll work, you find a balance. And whatever it produces, I git add -p one by one to make sure each chunk makes sense.

More work up front and some work after, but still saves time and brain power vs doing it all myself or letting it vibe out some garbage.

To a certain extent you are probably still not using it optimally if you are still doing that much work to clean it up. We, for example, asked the LLM to analyze the codebase for the common patterns we use and to write a document for AI agents to do better work on the codebase. I edited it and had it take a couple of passes. We then provide that doc as part of the requirements we feed to it. That made a big difference. We wrote specific instructions on how to structure tests, where to find common utilities, etc. We wrote pre-commit hooks to help double check its work. Every time we see something it’s doing that it shouldn’t, it goes in the instructions. Now it mostly does 85-90% quality work. Yes it requires human review and some small changes. Not sure how the thing works that it built? Before reviewing the code, have it draw a Mermaid sequence diagram.

We found it mostly starts to abandon instructions when the context gets too polluted. Subagents really help address that by not loading the top context with the content of all your files.

Another tip: give it feedback as PR comments and have it read them with the gh CLI. This is faster than hand editing the code yourself a lot of times. While it cleans up its own work you can be doing something else.

I agree. Brings me to the question though, how to deal with team members that are less experienced and use LLMs. Code review needs much more work then to teach these principles. And most of the time people won't bother to do that and just rubber stamp the working solution.

> Also, I make it work the same way I do: I first come up with the data model until it "works" in my head, before writing any "code" to deal with it. Again, clear instructions.

But then it's not vibe coding anymore :)

I have never seen a junior engineer make up API calls or arguments.