| ▲ | monsieurbanana 3 hours ago |
| I'm not anti-ai but I am firmly anti-make-people-read-llm-output-I-didn't-read-myself, and this breaks this. They also have a reward system that punishes people for ignoring the llm review (by pushing their submission lower in the queue). |
|
| ▲ | cromka 3 hours ago | parent | next [-] |
| > I'm not anti-ai but I am firmly anti-make-people-read-llm-output-I-didn't-read-myself, and this breaks this. This here might hit the nail on the head exactly in terms of what the biggest FOSSes gripe with LLMs with might be: not the fact people use it but rather how it hinders the communication. If you use it to help the write code, but you explain things in your own words or, at very least, redact and trim LLMs output, it's fine. If you, however, just drop a wall of text at someone and expect they act on its b/s, this is lazy to the point it borders with microagression. You wouldn't expect a distinguished professor explain basic maths to a student, and we shouldn't expect actual developers address Copilot's stupid, unredacted hallucinations or otherwise unrelated remarks. This works for patches submission, too: you should be outright banned for submitting stuff you didn't bother to double check and properly document. |
| |
| ▲ | close04 2 hours ago | parent | next [-] | | That is one of the biggest gripes for almost anyone I ever met who deals with AI generated content. Nobody really cares that someone else's work was generated with AI as much as they care that it's slop and they need to put in extra effort to work around the author's laziness. Lazy or incompetent colleagues always forced people around them to overcompensate and correct. AI increased their firepower and blast radius. | |
| ▲ | pluc 2 hours ago | parent | prev [-] | | If you could explain what you're doing why would you use LLMs to do it in the fist place? For speed over control? Explaining means you're effectively able to do the modification yourself, and I'm fairly certain that's a low percentage of all AI contributions. | | |
| ▲ | phoghed 2 hours ago | parent | next [-] | | Same reason I can explain how to clean my house but I pay someone to do it? I want the end result, I don’t want to do the whole process myself. | | |
| ▲ | lifeisloving 2 hours ago | parent [-] | | No, most AI users who contributor to OS do not actually know what they're doing. Its a problem. And no amount of analogies about house keepers will change that. |
| |
| ▲ | excannuk 2 hours ago | parent | prev [-] | | [dead] |
|
|
|
| ▲ | bayindirh 24 minutes ago | parent | prev | next [-] |
| Disclosure: I'm not a Debian Developer, yet I have worked on Debian derivatives and I'm somewhat familiar with internals of Debian from technical and human perspectives. While I'm perfectly on the same page with the critics of the LLM generated summaries, I'm a bit torn on the issue. For anyone not familiar with Debian ecosystem, there's a helper tool called Lintian. Lintian does all kinds of checks from package formatting, licenses to dependency management (which can detect both unnecessary yet included and necessary but missing dependencies via static analysis of your tool if necessary). It's expected that your package to be "Lintian clean" to be able to included in Debian with minimum friction and wait. Even though it's Lintian clean, your package also checked by teams which are severely understaffed, and composed solely of voluntary people. NEW queue also receives tremendous influx of packages (which can be considered a torrent, even). Since noon, there are 10 more ITP (intention to package) bugs open in my inbox. As a result, at one hand, these people need every help they can get because getting new trainees and getting them to learn the intricacies is hard, on the other hand LLM generated text rightfully generate negative reactions from people. However, giving developers an automated check of what looks right and what looks wrong is helpful, in my perspective. I wonder what people would have said it was a deterministic tool generating these reports and sending people these summaries as a "hey consider this feedback until a human has time to see your package". This is a hard question and we need to think before impulsive reactions. I even don't have all or any of the answers, but at least I have some insight why this is there. |
| |
| ▲ | monegator a few seconds ago | parent [-] | | one of the points is that LLMs are NOT deterministic so you can't run them yourself a thousand times and expect the same output. |
|
|
| ▲ | monsieurbanana 3 hours ago | parent | prev | next [-] |
| I guess I have to temper my own statement a bit. I do see situations where I would generate and send automated LLM messages. Probably to reply to generated unsolicited messages or pull-requests. But to upload to debian new's queue you already need to have pre-approval, it's a different situation from e.g. a public git repository that anyone can push to. |
| |
| ▲ | InsidiousWombat an hour ago | parent | next [-] | | But there is also way to throttle that. Have AI go thru submission guidelines, compare it to PR and have it shut up when most (not all! AI will always give a reason if you ask it to check something) of it is correct, and only answer when the PR itself doesn't comply to the project rules | |
| ▲ | g-b-r 3 hours ago | parent | prev [-] | | > where I would generate and send automated LLM messages. Probably to reply to generated unsolicited messages or pull-requests. Why would you use an LLM instead of a fixed message? | | |
| ▲ | monsieurbanana 3 hours ago | parent [-] | | To help classify between human and llm-generated ones? I don't know, and you're probably right than in this case a fixed message would be better. Another situation where I would use llms to generate messages is if we've mutually agreed to do that. For example automated review messages at work, if we've agreed that it would be a part of our workflow. A good rule of thumb might be unsolicited vs solicited llm messages. | | |
| ▲ | tetha an hour ago | parent | next [-] | | And even with a mutual agreement - default LLM output tends to be very hard and inefficient to read. We've spent quite a bit of time at work to setup a few writing skills for Claude. For example, mine is a technical writing skill based on the old book "The Art of Scientific Writing": Front load information as much as possible - the first paragraph of a document should be a summary, the first sentence of a paragraph should summarize. Short sentences building on each other. Always show evidence. Someone else spent quite a while optimizing a standardized PR description template, with a section for intent, expected changes, risk, rollout coordination and such. That made the slop a lot more tolerable. | |
| ▲ | g-b-r 2 hours ago | parent | prev [-] | | Ok, for unattended replies indeed a fixed message is much better; you can't predict what an LLM will output, and a certain impersonal message is better than a personalized one that might be completely senseless. If you mutually agreed of course it's fine |
|
|
|
|
| ▲ | SOLAR_FIELDS 3 hours ago | parent | prev | next [-] |
| Yeah AI is the relevant stand in but ultimately this is about one group of people being lazy and trying to shift more work onto another group of people. It just so happens that they are using AI to do it |
| |
| ▲ | account42 3 hours ago | parent | next [-] | | I don't think it "just so happens". AI enables that behavior at great scale and arguably even encourages it. | |
| ▲ | agwa 2 hours ago | parent | prev [-] | | You can disagree with their approach, but I don't think it's fair to call the DFSG team, which is comprised of unpaid volunteers, "lazy". They have to vet every brand-new package that enters Debian, and their queue has an infamous history of being a bottleneck in the Debian project. Their time is certainly more valuable than the the time of the individuals uploading new packages. | | |
| ▲ | thesuitonym 28 minutes ago | parent [-] | | I understand your confusion, but here's the truth of the matter: None of this has to be done. If anyone on the DFSG team does not want to do this work the answer is simple: Just don't do it. | | |
|
|
|
| ▲ | vitorsr 37 minutes ago | parent | prev | next [-] |
| Maybe it's an unpopular opinion, but a concise report from one of today's frontier models is not something I would be offended by, even if nobody read it beforehand. I guess the fact that there is a nontrivial cost to it also eases the existential angst of neither being deterministic (Lintian) nor having been produced by a person. |
|
| ▲ | gspr 2 hours ago | parent | prev | next [-] |
| > They also have a reward system that punishes people for ignoring the llm review (by pushing their submission lower in the queue). This is what I fear will push its way into every part of society soon, too. "We're not forcing you to do your banking through our AI interface instead of a normal one, you're free to opt out, but multiple functions will be missing and we'll charge you a monthly fee." "You don't have to have an LLM grade your exam, but beware that the human grader will be overworked and grumpy for the extra task." |
| |
| ▲ | drdexebtjl an hour ago | parent [-] | | This is happening to job interviews right now. If you’ve applied to jobs recently, you must have seen the companies that transcribe and summarize interviews using AI (and probably share data across their clients in an indirect, technically legal way). You can opt out, of course, but do you really want to be the one candidate for which the interviewer needs to take notes manually? And they just spent a lot of money on this hiring AI tool, will they hire the one candidate for which they couldn’t use it? | | |
| ▲ | solyin 14 minutes ago | parent | next [-] | | > transcribe and summarize interviews using AI That's cute. Yo, people be having full on AI-led interviews! That is to say, there's the human interviewee, and an AI, testing the human. Asking them questions and having a conversation that is definitely recorded. | |
| ▲ | gspr 44 minutes ago | parent | prev [-] | | So I've heard. I also know that several doctor's offices here use AI transcription. I'm sure I have a legal right to demand that they be switched off (I'm worried about privacy and accuracy here), but then I've got to be an asshole and increase the workload on my well-meaning doctor who probably had no say in any of this. This is gonna get so much worse. It's like the smartphone revolution on steroids – and we've seemingly learned nothing! We're in dire straits. | | |
| ▲ | wccrawford 6 minutes ago | parent [-] | | I went through a lot of medical stuff a few months ago, and every single doctor asked me if they could use the device for that instead of typing it into the computer themselves. One of them explained that it allowed them to focus their whole attention on me, instead of partially on the computer for the inputting of the data while we talked. I'm surprised that your doctors weren't asking, and instead were just assuming. |
|
|
|
|
| ▲ | throwawayffffas an hour ago | parent | prev | next [-] |
| Eh, you read compiler and linter output don't you? Nobody read that before you did. Sure LLMs are not nearly as rigorous about their output but that doesn't mean it's entirely not useful. At a minimum you can have your LLM summarize the other LLMs output and go from there. |
| |
| ▲ | johannes1234321 44 minutes ago | parent [-] | | Compiler output is somewhat deterministic and there is a logical path (even if caused by a compiler bug) and not dealing with it will definitely(!) lead to a broken product. Also at least I rarely read the full output, but I have an intuition, as the structure is consistent, what to look for in the output. LLM output may or may not be relevant, may or may not tell me something new and by the way it is structure is often hard to quickly oarse for the relevant piece in a wall of text. |
|
|
| ▲ | pantalaimon 3 hours ago | parent | prev [-] |
| Do you share the same opinion about script generated (pre-LLM) review comments? |
| |
| ▲ | monsieurbanana 3 hours ago | parent [-] | | It's disingenuous to compare LLM outputs to script generated ones. They have more differences than similarities. A script is reproducible, and fixable. They won't waste your time hallucinating issues. Note that this is different from "being wrong". A script can be wrong, and you can spend time debugging it, but that time is not wasted. You can fix the script, remove wrong rules, whatever. Also, nothing exists in a vacuum. Script-generated review comments were never a widespread societal issue, LLMs unfeterred content generation is. | | |
| ▲ | simias 3 hours ago | parent | next [-] | | It's a bit insane how it seems that "deterministic and reproducible", what I would consider to be a cornerstone of good software development, has apparently become an afterthought in the age of LLMs. All that for the sake of perceived convenience. Not even convenience at times, I think that for some the amount of technical-sounding text LLMs can output on demand is comforting. It feels serious. It feels like it's doing something important. Meanwhile the rest of us have to deal with these low-signal/high-noise walls of text in commit messages and PR comments. Or ask another LLM to summarize them, I suppose. "Doing and undoing, it's all work", as we say in French. | |
| ▲ | Grombobulous 2 hours ago | parent | prev [-] | | Is the hallucination or “being wrong” the only major problem? Because if that’s the case I can’t imagine that problem will still be around in ~1-5 years. I think it’s actually very similar to a script. It takes an input and produces an output. Certainly it has more variation than a traditional script, but it's generally going to return a very similar answer when the same input is given repeatedly, like when you ask an employee at a restaurant where the bathroom is. The verbiage may not be the same every time but the answer is always the same. |
|
|