Remix.run Logo
▲ malisper an hour ago

Yeah, that was a mistake. We should have waited until it was more complete before submitting it.

We were just really excited about what we had built and how fast it was that we wanted to submit it.

▲sondr3 an hour ago | parent [-]

This absolutely does not pass any smell test, by your own admission you included the test files "by accident", could only barely pass the minimum of SQL for the benchmarks, included environment variables for "internal benchmarking" and a preview version you cannot immediately release because it might contain sensitive data? What kind of sensitive data are you putting in a Postgres slop fork in such a way that you cannot immediately release it, besides obvious admissions of cheating? This all just reeks.

▲malisper 33 minutes ago | parent | next [-]

> This absolutely does not pass any smell test, by your own admission you included the test files "by accident"

If you don't believe you can run the binary yourself. You'll see the numbers match the ClickBench numbers and that those files are dead code during the benchmark

▲setr 32 minutes ago | parent | prev [-]

I mean, if the now-released open source variant performs roughly the same, and has no weird behavior in it… the matter is resolved, no?

They’ve already submitted the best possible corrective step for any sus behavior they previously did.

It doesn’t even really matter if the preview binary was in fact cheating, if the beta binary meets the benchmark results always. It’d be like cheating on a test you were gonna score 100% on anyways.