| ▲ | okeuro49 2 hours ago | |||||||||||||||||||||||||
It looks like quite a lot of analysis went into the rewrite https://bun.com/bun-unsafe-audit If the tests pass, then why not accept the rewrite? An interesting article of Prisma using the rewrite: | ||||||||||||||||||||||||||
| ▲ | zarzavat an hour ago | parent | next [-] | |||||||||||||||||||||||||
Just think about what "tests pass" means for a rewrite. If you rewrite from language A to language B then any unit tests have to be rewritten during the rewrite. So either "tests pass" does not include unit tests or unit tests were rewritten probably by the same AI that is doing the rewrite! | ||||||||||||||||||||||||||
| ▲ | JCTheDenthog 2 hours ago | parent | prev | next [-] | |||||||||||||||||||||||||
Given that even the unsafe audit article appears to be written by AI that doesn't seem like much thought to me. | ||||||||||||||||||||||||||
| ||||||||||||||||||||||||||
| ▲ | adamddev1 an hour ago | parent | prev | next [-] | |||||||||||||||||||||||||
People are forgetting that "if the tests pass" is not a good measure of reliability. CS 101 from Dijkstra: tests cannot prove the absence of bugs. | ||||||||||||||||||||||||||
| ||||||||||||||||||||||||||
| ▲ | applfanboysbgon 2 hours ago | parent | prev | next [-] | |||||||||||||||||||||||||
> quite a lot of analysis went into the rewrite Literally just prompted for an LLM to review it and asked for a fancy presentation. That is not "quite a lot of analysis". That is anything but. > If the tests pass, then why not accept the rewrite? Because (1) tests passing are absolutely not a guarantee that no regressions were introduced in a change, and (2) even if they were, those tests are the result of thousands of hours of human labour, which is all well and good for the codebase as it currently exists, but who is going to be writing the tests for the 1m loc repo of unread code in the future? Unless you've proven that specifically LLM-generated tests can prevent all possible regressions, you're condemning the future of the project because nobody will be able to continue writing robust tests. | ||||||||||||||||||||||||||
| ||||||||||||||||||||||||||
| ▲ | theturtletalks an hour ago | parent | prev [-] | |||||||||||||||||||||||||
[dead] | ||||||||||||||||||||||||||