ActionHank 5 hours ago | parent | next [-]

Should be outright banned as a precaution.

Two possible paths lead to their cheating code getting into a release. Either it was added by coding agent and no one looked at it, bad, but worse would be that a person was aware of it and approved - that's malicious.

Either way you don't want to be platforming the product because neither shows any degree of responsibility.

▲

malisper 3 hours ago | parent [-]

Or the third explanation is we didn't cheat: https://news.ycombinator.com/item?id=50209450

▲

fatty_patty89 2 hours ago | parent | next [-]

where's the sourcecode for 0.4-preview?

	▲	malisper an hour ago \| parent [-]
		I see what I can do. I'll need to audit it to see if we can release it

▲

LtWorf 3 hours ago | parent | prev [-]

https://news.ycombinator.com/item?id=50210444

▲

throwaway894345 2 hours ago | parent [-]

That comment doesn't evidence cheating, it reiterates that they submitted the result prematurely which was already admitted in the comment that your link was responding to.

EDIT: removed a lament that people don’t read comments they respond to, in case this violates the site rules.

▲

shawabawa3 2 hours ago | parent | next [-]

So, the charge was that they submitted a build specifically tuned for clickbench

The defense was that it wasn't specifically tuned for clickbench, it was just an early incomplete build that happened to only support exactly and clickbench queries and nothing else...

Doesn't feel like much of a defense to me. At best it's benchmaxxing if not clear outright cheating

	▲	malisper an hour ago \| parent \| next [-]
		> The defense was that it wasn't specifically tuned for clickbench, it was just an early incomplete build that happened to only support exactly and clickbench queries and nothing else. ClickBench tests a very narrow set of SQL: scans, aggregations, sorts, and filters. It ends up being a great starting point for building a query engine before layering in more functionality
	▲	throwaway894345 2 hours ago \| parent \| prev [-]
		> The defense was that it wasn't specifically tuned for clickbench, it was just an early incomplete build that happened to only support exactly and clickbench queries and nothing else... According to both the ClickHouse founder who was levying the accusations and the pgrust creator, the submission worked for many other queries it just didn’t have full SQL support.

▲

LtWorf 2 hours ago | parent | prev [-]

Insinuating that I didn't read is against this website's rules.

Using a throwaway account to reply is suspicious to say the least.

	▲	throwaway894345 2 hours ago \| parent [-]
		> Insinuating that I didn't read is against this website's rules. Fair enough, I edited my comment. > Using a throwaway account to reply is suspicious to say the least. My account is older than yours and has ~7x the activity. Maybe you mean “using an anonymous account is suspicious”?

▲

sondr3 6 hours ago | parent | prev | next [-]

Hope the ClickHouse maintainers stand their ground and do not let pgrust back in until it and the maintainer has matured more.

▲

Cyph0n 4 hours ago | parent | prev | next [-]

Love the confidence and complete lack of acknowledgement of the initial fuck up.

▲

Zookaz 5 hours ago | parent | prev | next [-]

Maintainer of pgrust trying to get it merged in is insane

▲

dorianmariecom 5 hours ago | parent | prev [-]

written with claude lol

▲

ncruces 4 hours ago | parent [-]

Did they ask Claude "don't cheat" this time round?

	▲	ModernMech 2 hours ago \| parent [-]
		You joke but when benchmarking with ai this is important. When you tell it optimize X, it will do so at any cost, including cheezing benchmarks. So you have to make sure you tell it what it can’t optimize along with what it can.