|
| ▲ | entrope 2 days ago | parent | next [-] |
| The regressions got fixed before a release. Do you count your internal integration branch as "shipped"? 0.02 defects per KLOC is a low density, even considering that it was "just" a port to a new language and before giving credit for the existing bugs it identified and fixed. That defect density will probably only generalize to other projects that have similarly extensive regression test suites, but it's a proof of what LLMs can do. |
| |
| ▲ | SpicyLemonZest 2 days ago | parent [-] | | The source article says that "Large code migrations are a particularly effective use case for these advanced models", and the specific advantages suggest to me that the results won't generalize at all. I can't think of anything other than a large code migration that would allow for thousands of independent workstreams against a sufficiently detailed spec to stop agents from going completely off track. |
|
|
| ▲ | jryle70 2 days ago | parent | prev | next [-] |
| They didn't ship 19 regressions over 2 weeks. "Merged" doesn't mean "shipped". Their customers have probably seen none of those because they've all been fixed. Such a terribly bad faith take. I feel upset reading this more than I should have. |
| |
| ▲ | torginus 2 days ago | parent | next [-] | | Yeah, it sounds more like they found 19 regressions after the migration so far. | |
| ▲ | SpicyLemonZest 2 days ago | parent | prev [-] | | On what basis do you say their customers probably didn't see them? I'm not intimately familiar with Bun, but I work with people who try to do software development in this way, and they routinely ship regressions to customers. They believe that this is because we're not AI-enabled enough, and if only we had the right loops they'd stop shipping bugs; I think that it's not about the loops and the bugs reflect their lack of understanding about what they're trying to do. | | |
|
|
| ▲ | JackSlateur 2 days ago | parent | prev | next [-] |
| I guess even anthropic fails to not get slop from ai .. |
|
| ▲ | IshKebab 2 days ago | parent | prev | next [-] |
| Umm yeah because you didn't rewrite a million line codebase in Rust in two weeks. Obviously "regressions" is not a great measure (how serious were they?) but 19 seems pretty decent to me. |
|
| ▲ | scrollaway 2 days ago | parent | prev | next [-] |
| Migrating a million lines of code is how many human-engineer-years worth of work? Some quick fermi estimates place it at between 15-100 years depending on how much automated tooling is used (without AI). It really blows my mind how some AI naysayers like you fail the absolute basic common sense test of comparing the comparable. If you, without AI, merged nineteen regressions in 2 weeks, how much work has actually been done? How many features did you rewrite? How many refactors did you make and how difficult were they? Insane... |
| |
| ▲ | nozzlegear 2 days ago | parent | next [-] | | Without AI, the migration would never have happened. People with stake in the code wouldn't have invested time in it, and would've taken the much more sane and logical route of improving their Zig codebase. | | |
| ▲ | scrollaway 2 days ago | parent [-] | | And who put you in charge deciding that improving their zig codebase is the One True Way To Do Things? Fuck, man, this used to be hacker news, a place where curiosity is rewarded. | | |
| ▲ | nozzlegear 2 days ago | parent | next [-] | | Bizarre response. I apparently need to draw attention to the fact that I didn't claim to be in charge of Bun's codebase; I was merely pointing out the obvious conclusion, what would have happened before everyone collectively lost their minds and their focus to the AI craze. | |
| ▲ | a day ago | parent | prev [-] | | [deleted] |
|
| |
| ▲ | rurban 2 days ago | parent | prev | next [-] | | Exactly. My AI used to tell me some human costs of work for this feature. Like 2 months. I told start doing it, and it finished within 2 hours. Insane | |
| ▲ | SpicyLemonZest 2 days ago | parent | prev [-] | | It's precisely because I'm not an AI naysayer that I reject the premise of the comparison. My job is not to produce as much code as I possibly can; my job is to produce features that provide value to customers in a high quality and regression-free manner. To accomplish this I use safer AI-enabled development strategies, which produce fewer lines of code and involve more manual review but also do not result in 1 regression per calendar day. | | |
| ▲ | scrollaway 2 days ago | parent [-] | | Again, you are comparing things which are not comparable. "Working on a feature" and "Working on a language-to-language port of 1M LOC" are NOT the same thing. You're also still bringing up "regression per day", while ignoring the fact that without AI, this would have taken easily ~4000 days. Which is 1 regression per 0.003 calendar day. Not the same. And you still aren't addressing sibling comments reminding you that these aren't shipped migrations. I'll second the comment asking you whether your code is always perfect first-shot, because if we're going to count those bugs, we might be well above your 1/day number. | | |
| ▲ | well_ackshually 2 days ago | parent [-] | | No, without AI, it wouldn't have been done, which would have led to exactly zero bugs added by a needless rewrite. |
|
|
|
|
| ▲ | Salgat 2 days ago | parent | prev [-] |
| Regressions is a meaningless metric to me. Every app has bugs, and most are either rarely encountered or aren't breaking the app. All I care about is interruption of service. |
