Remix.run Logo
cyphar 5 days ago

Code reviews (especially internal ones) generally assume that the person writing the original code has an idea of what they are doing and are designed to catch mistakes that humans might make. Just because they probably work to improve codebases with human submissions doesn't mean that they are good enough filter for LLM-generated code that the submitter doesn't sufficiently understand and has submitted without their own review. Same goes for CI and testing.

This reminds of some of the comments made by reviewers during the infamous Schön scientific fraud case. The scientific review process is designed to catch mistakes and honest flaws in research. It is not designed to catch fraud, and the evidence shows that it is bad at it.

Another applicable example would be the bad patches fiasco with the Linux kernel. (And there is going to be a session at the upcoming maintainers' summit about LLM-generated kernel patches.)