Okay, but to return to replications, publishers could incentivize replications by linking replication studies directly on a paper's website location. In fact, you could even have a collection of DOIs for these purposes, including for datasets. With this point in mind, what I find depressing is that the journal declined a follow-up comment.

But the article is generally weird or even harmful too. Going to social media with these things and all; we have enough of that "pretty" stuff already.

▲

dgxyz 2 hours ago | parent [-]

Agree completely on all points.

However there are two problems with it. Firstly it's a step towards gamification and having tried that model in a fintech on reputation scoring, it was a bit of a disaster. Secondarily, very few studies are replicated in the first place unless there is a demand for linked research to replicate it before building on it.

There are also entire fields which are mostly populated by bullshit generators. And they actively avoid replication studies. Certain branches of psychology are rather interesting in that space.

▲

jruohonen 2 hours ago | parent [-]

> Certain branches of psychology are rather interesting in that space.

Maybe, I cannot say, but what I can say is that CS is in the midst of a huge replication crisis because LLM research cannot be replicated by definition. So I'd perhaps tone down the claims about other fields.

	▲	dgxyz 8 minutes ago \| parent [-]
		Another good example that for sure. You won't find me having any positive comments about LLMs.