Remix.run Logo
▲ spongebobstoes 2 hours ago

thanks for your explanation. I'd love to see a concise explanation of the difference in benchmark performance between -preview and -beta

that would help clear the air, if the full-sql version has similar performance to the partial-sql version

▲malisper an hour ago | parent [-]

The new general version is actually faster than the version accused of cheating.

The architecture of the query engine is:

  SQL -> IR -> Assembly
-preview had the full IR -> Assembly pipeline built out, but only part of the SQL -> IR part. As long as queries compile to the same IR, they're just as fast. We've shipped some more optimizations, so they got even faster.
▲infogulch 35 minutes ago | parent [-]

If it delivers on being faster than the version accused of cheating with reasonable cache settings and no SQL feature limitations, I would say that resolves the accusations, regardless of the source code of -preview.

The remaining lesson here is that one should temper one's excitement with pre-release software, whether the software was built with AI or not (rather, especially if its built with AI). The preview results would have been more appropriate to announce on a social media platform than as a PR to the ClickBench repo.