Remix.run Logo
▲ verdverm 7 hours ago

> Because we have test result data that PROVES it to be the case.

pgrust thought this too, but then another human discovered claude had hardcoded benchmark query plans.

Now pgrust has a "test" [1] to make sure the Ai doesn't do this again (unless it decides this test is bad for the current goal)

I have a growing list of these, another notable one is OpenCode having a security regression [2] because the Ai redesigned configuration loading for v2.

Real verification seems beyond transformer capabilities.

[1] https://github.com/malisper/pgrust/blob/38634b408dea373f3944...

[2] https://github.com/anomalyco/opencode/pull/54921