| |
| ▲ | flohofwoe 9 hours ago | parent | next [-] | | You sound like a webdev, e.g. the "let's just install 1000s of unvetted external dependencies" is nowhere else nearly as extreme as in web development land. | | | |
| ▲ | EvanAnderson 9 hours ago | parent | prev [-] | | I don't think that's accurate. Until recently the code we rely on wasn't blindly trusted. People wrote and QA'd it. We may not have reviewed it personally, but that's one of the functions we outsource to software maintainers or "manufacturers". I'm not "blindly trusting" code on my computing devices. I'm trusting the vendors / maintainers to do their job. Until very recently the norm has been that the vast majority of code had human eyes and hands on it. Edit: The owners of those human eyes and hands had some type of accountability (either reputationally, in the case of free/open-source software, or occupationally, in the case of proprietary software). The LLM has no accountability as to the output it generates. The companies who make the LLMs also seem to have very little accountability, too. We've assumed a "blame the victim" stance when people use LLM-generated output in some inappropriate ways (legal briefs with "hallucinated" citations, articles "written" by LLMs). Whether that's the right location for accountability to be placed isn't for me to say, but that seems to be how it is. I'm not sure that we're applying accountability to LLM-generated code in the same way we are for, say, the LLM-generated legal brief. | | |
| ▲ | jadar 9 hours ago | parent | next [-] | | A compiler is not an LLM, and I do not want to equivocate, but there are aspects of similarity. We do not assume people read the bytes of machine code to ensure it’s correct — there could be mistakes. We also write and run automated tests to ensure the code outputted from a compiler and an LLM behaves correctly. At some point, we won’t have to literally read every byte of code that comes out because we have a reasonable assurance that it’s correct. | | |
| ▲ | EvanAnderson 9 hours ago | parent | next [-] | | That we don't have to heavily scrutinize the code generated by compilers is a result of the huge amount of human toil that went into the compilers and test suites. Compilers are deterministic mechanisms so tests can be constructed. LLMs, at least as they're currently constructed, aren't deterministic (the whole "temperature" thing). I don't see how to build a mechanistic test for something that has non-deterministic output. It feels a little bit like solving the halting problem. I have no doubt we'll move away from human code review. The idea of large amounts of software edifice being built upon foundations that no human has reviewed or, perhaps even understands, is horrifying to me, though. | |
| ▲ | flohofwoe 8 hours ago | parent | prev [-] | | Compilers also usually give you the same output for the same input. And fwiw I do spend quite a lot of time reading compiler output to check that it's not doing something stupid or unexpected (usually as part of optimization work). Also this sort of 'technological whataboutism' really isn't helpful, compilers are entirely different from LLMs. I agree that it doesn't make much sense to read or review LLM output in detail, but I also don't plan to use LLM output for anything important or mission critical. That would be irresponsible. | | |
| ▲ | AshamedCaptain 8 hours ago | parent | next [-] | | In addition, when we usually say that triggering undefined behavior on C can start a game of Tetris or format your hard disk we're usually joking (or at least exaggerating), i.e. the most common failure conditions of compilers are really limited in scope, and very likely caught by whatever testing mechanism you use for the software. No such limits for LLMs where losing all your files is about par for the course for everyone who uses them regularly. | |
| ▲ | pjmlp 4 hours ago | parent | prev [-] | | JIT compilers certainly do not give the same machine code for the same input. The actual machine code depends on several parameters, and it is very hard to replicate them, hence why many devs get benchmarks with JITs wrong. Additionally, compiler optimisation passes with machine learning is starting to be a thing, yet another way how the machine code differs for the same input across compiler executions. |
|
| |
| ▲ | tossandthrow 9 hours ago | parent | prev [-] | | > Until very recently the norm has been that the vast majority of code had human eyes and hands on it. What quality does it have that humans had eyes on code? Elite teams likely still produce code of better quality with a higher qa bar than agentic code. But that category is dimishing everyday. The core point is that so much trust is reduced to "Joe in cubicle". Also a lot more than what he can carry. Most of the code that is being executed on your behalf is far from written by elite teams. | | |
| ▲ | flohofwoe 9 hours ago | parent | next [-] | | It has nothing to do with 'elite teams', it has to do with decades long and very careful maintenance. It's too early to say whether LLM generated projects will ever reach that sort of maturity, most examples I've seen so far are basically "fire and forget". But lets talk again in one or two decades, maybe there will be counterexamples of successful open source projects which will be just as well llm-maintained as human-mainained. But I suspect that to reach that sort of maturity, the resulting human effort will be mostly the same (e.g. not much of a productity win - except maybe on the 'edges', e.g. maintaining the test suite, documentation, helping to analyze bugs..., e.g. these are examples where LLMs are genuinely useful and where plagiarism hardly matters). | | |
| ▲ | tossandthrow 8 hours ago | parent [-] | | It really is not. There are some areas that are critical and where software developers carefully will detail stear the work. But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents. | | |
| ▲ | flohofwoe 7 hours ago | parent [-] | | > But the vast amount of software written, react components and rest endpoints, are very ripe to be entirely written by agents. In that I agree, nobody should be forced to write React code manually, that's almost a human rights violation ;) REST endpoints (and the code talking to those endpoints) should be code generated anyway though, no need for LLMs, and instead of human language prompting, a precise IDL should be the spec and basis for a mechnical code generation process. That problem was solved decades ago with much more pedestrian technology. E.g. it basically comes down to "it's fine to use LLMs for software that shouldn't have been written in the first place", and funny enough that's where LLMs are really good at: creating software that has been written a million times before with only minor variations, and doing this type of work manually (cranking out one cookie cutter React webpage or REST API after another) is essentially what's called 'bullshit jobs' (which bring food on the table though, but that's another topic). PS: > and where software developers carefully will detail stear the work. ...I think the further a project evolves, the less this "detailed stearing" will be any more productive than doing the same without LLMs. The older a project, the more the work shifts from implementation to decision making, and in most cases the result of that decision is just a very tiny code change. I already see cases in my daily work when I use 'agentic workflows' where a tiny change takes longer and involves more 'collatoral updates' then just fixing that one frigging line of code by hand like in the olden days, and for LLM-generated code bases I really do prefer to not mix LLM and manual work, I think that's the worst of all options. |
|
| |
| ▲ | EvanAnderson 8 hours ago | parent | prev [-] | | > What quality does it have that humans had eyes on code? The human who had their eyes and hands on the code has accountability. I think how accountability is going to work, in the case of LLM-generated code, is still an open question. (I dropped-on an edit to the parent comment to this effect, too.) | | |
| ▲ | tossandthrow 8 hours ago | parent [-] | | > The human who had their eyes and hands on the code has accountability. They are not. You can not go go back to a laid off person / one who quit and keep them accountable. So this is absolutely not the case. Devs are not accountable for their code. | | |
| ▲ | EvanAnderson 8 hours ago | parent [-] | | You're being reductive or just obtuse. The company that employed the developer ultimately holds the accountability in the marketplace. The employed developer maintains (or loses) their job because of their accountability to their code (or, at least, they should). There's an economic incentive for all parties involved. In the free/open-source world the incentives aren't economic, but they're still there. | | |
| ▲ | tossandthrow 7 hours ago | parent [-] | | > You're being reductive or just obtuse. > or, at least, they should Seems like you are the obtuse one here, and you appear to know it. | | |
| ▲ | EvanAnderson 7 hours ago | parent [-] | | I don't get how saying companies should hold their employees accountable, which doesn't always happen, is somehow obtuse. I give up. I feel like you're a robot designed to waste my time. | | |
| ▲ | tossandthrow 6 hours ago | parent [-] | | Fair enough, that's on you. You are talking from an ideal point of view. You want the companies to hold employees responsible. But that is is not how it work. Why you appear obtuse. Eg. Look at th3 therac25 case. No developers was held accountable. We have spend more than a decade remove accountability from indiviauls. Using limited liability, insurance, and workers protection. Accountability is the last reason why we need humans over agents. | | |
| ▲ | EvanAnderson 3 hours ago | parent [-] | | Sorry for the exasperation. At the start of this you said: "For 99.999% of people, it is literally kthxbye on all code they execute on all their devices." I think that's inaccurate. The vast majority of code running on "all their devices" is code made by employees of companies being held accountable through traditional industry methods, or free.open source projects where reputational integrity was at stake. Those developers have been held accountable, for some value of accountable. Maybe there's less value in human accountability than I think there is. Only time will tell. That's a different conversation. The code running "for 99.999% of people" is not "literally kthxbye" LLM-generated code without someone behind it holding accountability. Maybe it will be in the future, but it's not now. |
|
|
|
|
|
|
|
|
|
