I never care about AI usage disclosure, because I don't believe that human produced code is necessarily better than AI produced code, unless it's someone I personally know.

People need to be responsible for code they commit and push anyways. This has never changed. Whether the code is written by hand, by their cat walking over keyboard, or by AI, is not my concern.

A project's code quality can decline for all kinds of reasons. I don't think it's productive to laser-focus on whether it's produced by AI or not. That's a distraction. If a person just want to find excuse to criticize AI, and another person wants to fight back and defend AI, sure, go for it. But that's not how you would want to assess a project's code quality.

> You’re just going to make people disable Claude attribution on their commits to avoid drama.

People should be doing this regardless of drama. No reason to provide free advertising for trillion dollar corporations. Generated-by trailers are only relevant when contributing to third party projects, in that case disclosure is polite.

Is that a bad thing? I mean from the perspective of Anthropic's marketing department sure, but if agents are just another type of tool in developer's tool belt - as I see people recently like to claim - attribution feels kinda weird. In the end it is the developer who is responsible for their commits.

I mean, I don't think commits are the place for tool attributions. I want to know what the change was, I'm not really interested in your tool selection (put that in the PR if it's relevant). It'd be just as irrelevant to see "written on my macbook in neovim"

I think it will be funny to watch people lose their collective minds when open source maintainers start requiring llm use.

This idea that the community can try to pressure an open source maintainers about the tools they use based off of kneejerk political reactions is so offensive.

Let's go the opposite way: "sorry I'm closing this pr because it didn't use an llm."

Your first and second points seem to contradict each other because if all of the bugs for 3.4.1 should be attributed to 3.4.0, that pushes the timetable back even further that unattributed LLM commits would have to have been being committed to the project, which just makes your point even more absurd.

Which brings me to my overall response, which is that there is absolutely no evidence, and nothing even intimating this hypothesis, that LLM commits were secretly being added to earlier releases before they were attributed, and that's why the rate of bugs is higher. There's no reason to think that it's an unreasonable thing to think, and there's no evidence for that whatsoever unless you beg the question and assume that higher bug counts must automatically indicate AI involvement, which is just circular reasoning. You're essentially just making up a hypothesis out of thin air to preserve your point.

Regarding your third point, that one's fair, but I've done the analysis and I can put it up if you want, as to how long it usually takes to find bugs and how far through the release cycle we are for each version.

You can use LLMs in multiple ways, from very hands on to make local changes to completely hands-off.

I've seen plenty of code that was LLM generated but the commit message itself did not have the co-author attached to it. This only seems to happen when someone's interface to the codebase is completely though Claude/Codex/..., and those are usually the most verbose commits, and yet they say the least, because they just summarize the code changes, not the why.

On the other hand I've seen developers using Claude as a tool. They have VSCode open and a terminal window with Claude and go back and forth, ensuring they write correct code, and leave the plumbing to Claude.

So maybe the author of the code started off small and it grew over time?

That which is asserted without evidence can be dismissed without evidence. This is more evidence, and of greater rigor, than was used to make the assertions. That's good enough for me. If someone wants to actually do the work to support the original claims with better evidence, great. I'd love to see it. Until then, I'm going to not worry about this issue.

But it is fair. Up to this point I have yet to see anyone say they did an analysis of the code and found X regressions of Y severity. All they say is "there are more bugs because LLM". This analysis, which you can verify yourself if you wish, says "the bugs [number of] are pretty average even with LLM", which is a direct response to that. If you'd like a more nuanced analysis you're welcome to do one and share the result, if you're so inclined.

Thank you!

IMO "bugs per commit" is even worse than that, because, in addition to what you say, it also hides the extraordinary spike of commit activity of a project that had previously been stable. [0]

It is the exact metric you'd choose if you wanted to make the current situation of rsync look like not a big deal.

[0] https://github.com/RsyncProject/rsync/graphs/commit-activity

I've now resolved this. The new version, which should be live on GH Pages soon, uses — what I think is — a pretty good methodology for assigning severity to each bug, normalizes it to 0.0-1.0, sums that, and treats that as the total severity weighted bugs, then does the analysis based on that. It did not change the analysis in any material way.

Why don't you prove the bugs increased then?

Why is it that some unfounded claim is made and the onus is suddenly on the project maintainer to prove it beyond all doubt?

It should be on the person making the claim to prove it

There was no analysis of severity in all of the rage posting that occurred. The single point being pushed was "use of an LLM led/leads to more bugs". The author specifically states that's what they're addressing (blunt accusation -> blunt response).

"fewer"

That's not what I'm saying. What I'm saying is that if the criticism is referring to a broad set of metrics like bugs per release and number of commits that were made by Claude, then it's correct to look at precisely those things because that's what the claim is about.

AI + Interest != Expertise

I come to hn because I get very nuanced, informed information and glorious puns.

"Did cars increase traveling deaths?"

"Cars are just a tool. The drivers who piloted the vehicles and weren't careful enough [are responsible for the deaths.]"

If something's a bad tool that misleads people into doing bad work, it would be good to know that.

This tool is claimed to be able to find and fix bugs.

Please read the article.

The unsolicited security reports are the issue.

It is neat that Lobsters has this feature (and HN should too), and I'm glad you took a beat to explain it. I think you didn't need the last sentence, though.

I have done so! that was a misremembering on my part. first mention of Lobsters is now here:

> On Lobste.rs, in response to the Medium essay Tridge himself posted in response, finally some users like boramalper begin to actually ask for evidence one way or another:

FWIW, I asked ChatGPT to review the article just for my amusement. It's conclusion was:

"My honest assessment is that this is a competent calculation performed on a badly confounded measurement, followed by conclusions substantially stronger than the calculation warrants. It is useful as a rebuttal to “the Claude releases are obviously unprecedented disasters,” but not as evidence that Claude was harmless."

It very obviously is. "The Outlier Nobody Noticed" -_-"

So the original unfounded claim has 400+ comments because its perfect HN ragebait

The author provides evidence to the contrary and the HNers won't even engage with it instead just talking about the writing of the article in classic HN bikeshedding fashion.

How about after that we talk about the formatting of the website and the colors?

This site is really going down hill

Where is the accountability for your own opinions?

Are you guys only upvoting things that confirm your existing gripes?

Some notes on this:

- I used GLM 5.1 to help with the coding and math for this.

- However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions)

- Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical methodology would be justified for this very limited data set, while still giving as much information as possible.

- I know the website looks like we stereotypically consider vibe-coded websites to look, but I actually explicitly asked for that. The original HTML design looked like a website from 1995, and I just prefer how this looks. It's pretty!

This submission was heavily flagged, presumably because the article sounded like genai. But the article now says the following:

> After posting this on Hacker News and recieving almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice.

I've therefore turned off the flags and hopefully people can actually now discuss the claims/findings being reported.

> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?

Yes: https://news.ycombinator.com/item?id=48390931

So far it reintroduced several security issues and replaced the README.md.

Emdashes don't really tell you much anything these days tbh. Many languages use them regularly and those people often bring the habit with them when they write in English — me included. Plus I would imagine every major model has tuned them way down at this point due to the backlash.

I rewrote all the AI prose several hours ago with purely my own. I like em-dashes, and specifically use them with spaces as a habit. I don't know what to tell you.

That version has security fixes from the same day as the latest rsync release: https://ubuntu.com/security/notices/USN-8283-1

As usual, Ubuntu backported fixes and didn't upgrade to a new version. Whether or not they also backported regressions in edge cases that afflict the latest rsync, I don't know. Pinning the Ubuntu package may prevent getting further regressions, but is preventing you getting any future such backported security fixes.

Did you face any actual bugs or regressions? Or are you doing this just because of the bandwagon that's going around right now? Because until you can actually present an argument for why this release is worse than any of the others, which is precisely the subject of my post, then this is not an argument against my post at all. This is just a self-referential appeal to authority.