| ▲ | Zigurd 3 hours ago |
| I find myself pulled in two directions by the performance of coding agents: First, keeping them on a short leash seems sensible. I don't get failed results very often. When I first started using coding agents, this seemed not just sensible but required in order to not deal with a lot of hallucinated weirdness. Secondly though, it's pretty cool to vaguely wave your hands in a certain direction and get both user interface and business logic that's objectively nicer than I would've gotten from the effort I would've put into manually coding. Whenever I cautiously use a coding agent, I wonder what I am missing by not letting it cook. |
|
| ▲ | contravariant 3 hours ago | parent | next [-] |
| I think Conway's law applies. In an ideal situation your code mimics your organisational structure. The places where you let an agent do its thing is bound by the (implicit) interface your prompt creates. In an ideal situation this interface is a simple isolated thing in your application. If you just let it do its thing, expect a monolith. You shouldn't rely on it coming up with interfaces that mesh well with what you had in mind, and each of those interfaces represents a decision that it took, restricting future iterations of the code. So a monolith is not only what you can expect, but also the best you can hope for. If you want to let it cook, create more places for code that does a thing and you don't care how. |
| |
| ▲ | dcss_gardener an hour ago | parent | next [-] | | You can steer it a lot with implicit context too, like ants following pheromones. If you put an empty docs/ dir in your repo agents will put documentation there without being told. If you put one inline test, they'll write every test inline. If you seed a decision record they'll record their decisions. If you have a lowercase sql query they'll use lowercase sql. If you write a test for wcag AA contrast they'll make ui with wcag AA contrast. And so on. With some up front care you can create an environment where they mostly do what you want when you let them rip without oversight. As the codebase grows it becomes less likely they'll see any one example, and more rarely expressed ones can get missed. And then codebase predating LLMs or designed without this consideration you just have to accept what you get or wrangle them manually. | |
| ▲ | Ericson2314 3 hours ago | parent | prev | next [-] | | BTW the usual point of Conway's structure is to try to adjust the org / who is working on the software, so you don't get a bad software architecture. The "objective" nature of code is revealing a flaw in the human division of labor. | |
| ▲ | pjc50 3 hours ago | parent | prev [-] | | Oh that's a good insight. The agent counts as part of the "organization" for this purpose. | | |
|
|
| ▲ | nolok 3 hours ago | parent | prev | next [-] |
| I don't understand why that idea was adopted that IA had to be that "give one command have a perfect run and correct answer" thing when it's clearly not how it works and an eternal "the last % is worth 99%" issue. I know why the US frontier labs push it of course, you can't push a trillion dollar plus idea if you also say it should be a cheap commodity and the best and latest are now the answer, a bazillion mid levels are. But most work of value I do is with a swarm of "not the latest nor the brightest" models, with a smart one controlling the dance. Most decisions or review is better done through consensus. The myth that "if we keep going it will reach a point where it's always correct" seems so obvious to me, it doesn't work like that, for that matter we don't work like that. This is not a jab at it. I work with it. I love it. But I think the story imperative are a mismatch for the actual needs of most companies, and it's hiding between the "frontier needs" of special research & co, or the governemental needs. On the same line of thought: it's pretty obvious as great as these things are, no they're not going to "replace all employee", and no employee without them won't be able to compete either. They're a supercharger, a self multiplying smart assistant with capabilities in any field, and it feels so wrong to me that they've been presented in a complelty different way. |
| |
| ▲ | krrrh 2 hours ago | parent | next [-] | | > it doesn't work like that, for that matter we don't work like that. The parallels with self-driving cars are interesting. We demand that agents don’t just produce better code than most humans (which the frontier models do), but that they perform much much much better. | | | |
| ▲ | wat10000 2 hours ago | parent | prev | next [-] | | I recently tried some personal evals to see how some different models performed. I was trying Opus, Sonnet, Gemini 3.8 (this was pre-Argon) and a couple of others. I picked a pull request I'd recently made with heavy Claude assistance, one that seemed simple enough to be quick to solve but with enough subtleties that I could differentiate the results. Set up the agents to run clean without knowledge of the solution, and set them loose. To my surprise, none of them came close to producing an acceptable PR. Quality varied, but even the very best results were missing significant pieces. My takeaway is that the human in the loop is still massively important for many things. And that makes it really hard to come up with a useful benchmark. | |
| ▲ | Edward40 2 hours ago | parent | prev [-] | | [dead] |
|
|
| ▲ | rrr_oh_man 3 hours ago | parent | prev | next [-] |
| > objectively nicer than I would've gotten from the effort I would've put into manually coding I find that only works with topics I know very little about. |
| |
| ▲ | datsci_est_2015 3 hours ago | parent [-] | | This comment could be interpreted at least two ways, and I’m not sure which it is: - tongue in cheek: in topics you know very little about, you don’t have the ability to tell whether the LLM output is trash or not
- very serious: LLMs allow you to create things you could not have created before due to not having the knowledge
Sort of like a Dunning-Kruger or Gell-Mann amnesia. | | |
| ▲ | devmor 3 hours ago | parent [-] | | I don't see why both can't be true. LLMs are allowing people to create things with very little effort compared to before, and those creations are of questionable quality - indeterminable by the LLM-user due to a lack of experience. | | |
| ▲ | ygjb 3 hours ago | parent [-] | | So, basically, the same as before, but LLMs provide the creations of questionable quality at a faster rate :P I think the quality of the results vary with the rigor of the process. If you one-shot prompt a new service (even with Fable5) "build me an airtable competitor, no bugs", you will get some real code camp quality results, but a well defined set of prompts and a workflow that leverages multiple coding agents reviewing each others work, and implementing test cases at a relatively granular level yields radically different results. Couple that with a rigorous design/spec session, and multiple checkpoints (say at release milestone) to monitor drift from the original design makes the process a bit more intentional and measurable, while still being far faster than a typical engineer can build. | | |
| ▲ | yugoslavia4ever 2 hours ago | parent [-] | | Yes, but before you learned as you went, researching and finding out what good (or at least better) looked like; what learning is done when you wave your hand? |
|
|
|
|
|
| ▲ | embedding-shape 3 hours ago | parent | prev | next [-] |
| Not every problem should (or even could sometimes) be solved the same way. Are you planning to maintain the produced code for more than a year? No, then wave away. But if you're expecting others to depend on it, and you want to be able to continue evolving what it is, then a tight leash and full understanding of how it actually works, is very much needed, even if you don't actually type characters into an editor anymore. Otherwise you still end up with balls of spaghetti where each change takes longer and longer to work into the codebase. With a bit of planning, you can keep tackling changes and they keep taking the same amount of time more or less. |
| |
| ▲ | smallnamespace 3 hours ago | parent | next [-] | | The cost of AI is rapidly falling while its engineering capabilities keep rising. This implies that code is becoming more fungible, and you should expect any code you write today to mostly be replaced in the near future. The requirements will be more stable than the code. Focus on them instead. | | |
| ▲ | datsci_est_2015 3 hours ago | parent | next [-] | | Robust systems experience very little turnover in nearly every non-software discipline that involves systems. Why the hell would we insert risk into our system-building processes by needlessly turning over code? | |
| ▲ | Zigurd 3 hours ago | parent | prev | next [-] | | > The requirements will be more stable than the code. These kinds of changes to the way teams are formed and managed, and how products are developed is what really fascinates me about the impact of coding agents. Even if it works, for some definition of works, to dump a huge spec into a coding agent context window, I suspect this is where a lot of horror stories about agentic coding come from. Teams, and the management structure above them, are going to have to get used to more collaborative and iterative product definition and development in order to extract the most value from coding agents. | |
| ▲ | catdog 3 hours ago | parent | prev | next [-] | | > The cost of AI is rapidly falling while its engineering capabilities keep rising. Wouldn't count on it. There likely is a ceiling, we might or might not be near it. | | |
| ▲ | reddit_clone 2 hours ago | parent [-] | | Definitely with the cost. I strongly believe the price now is heavily subsidized and will shoot up in the future. (Not new. First hit is free..) |
| |
| ▲ | wtch98 3 hours ago | parent | prev | next [-] | | > The cost of AI is rapidly falling while its engineering capabilities keep rising. This implies that code is becoming more fungible, and you should expect any code you write today to mostly be replaced in the near future. I have code I wrote in 2006 which is still in daily use adding business value at my company. Why would it need replacing? | |
| ▲ | tempfile 2 hours ago | parent | prev | next [-] | | > This implies that code is becoming more fungible Code has always been fungible, in the sense that it could be refactored. > you should expect any code you write today to mostly be replaced in the near future Why on earth do you think this? It does not follow from your other points. Just because you could turn your business into a rube goldberg code juggling machine, why would you? | | |
| ▲ | skydhash an hour ago | parent [-] | | it's amazing that you could read a book like "The Design and Implementation of the 4.4 BSD Operating Systems" (1996) and still find it relevant to browse the code of OpenBSD and NetBSD, three decades later. While good software gets plenty of updates and additions, the core ideas barely bulge over time. A lot of churn is people playing around with no reason instead of solving the problem properly the first time. |
| |
| ▲ | alala-lala-lon 2 hours ago | parent | prev [-] | | [dead] |
| |
| ▲ | krrrh 2 hours ago | parent | prev [-] | | A shift is realizing that refactors go from something that was costly and infrequent to something that can happen inexpensively and frequently. It takes some rebalancing. I recently used agentic processes to bring a young codebase to a functioning state very rapidly. This forced discussions about design principles that were folded back into a short document in the repo which then led to a major refactoring plan, and a massive pruning of code that was written the week prior. There are solid engineering approaches in the codebase now that wouldn’t have been achieved without this discovery process. |
|
|
| ▲ | Edward40 3 hours ago | parent | prev [-] |
| [flagged] |