One thing I noticed on the CS PhD side of the house is because many researchers don't want others to easily build upon their work (for whatever reasons), they don't often release the source code/data required to quickly validate it. This is a recipe for shortcuts, errors, and even in the worst cases, fraud.

▲

anishrverma 3 days ago | parent | next [-]

Strongly agree. When code, data, and workflow details stay hidden, the system rewards claims more than verification. That is where shortcuts, irreproducibility, and worse can thrive. We need infrastructure that gives more credit to transparency and reusability, not less. That is part of what we’re building at Liberata if you’re curious: https://liberata.info/beta-signup

▲

BeetleB 3 days ago | parent | prev [-]

When I was in grad school, this was the norm across the board (engineering/physics). No one wanted to reveal their secret sauce.

Things have changed since, but in my time, if a journal required source code for publication, most of the professors in my department would not have published there.

	▲	cge 3 days ago \| parent [-]
		>Things have changed since, but in my time, if a journal required source code for publication, most of the professors in my department would not have published there. Even when they do require it, one of the problems for journals that require source code for publication is that there is little support for making sure that code actually works reliably. Reviewers often aren't obligated to look at code packages, and when they are, might not be expected to actually get anything running; they might not even have the resources to do so. I have done reviews where I have tried to get code to run, and oftentimes I feel like the code needs work not because of any malice, but simply because making a one-time code package that works, and continues to work, for others, over time, without updates, can be quite hard, especially when odd dependencies are involved. It's also not necessarily something related to normal reviewing of scientific content, but more things like insufficient dependency pinning, accidentally hard-coded paths, environment assumptions that worked for the authors but not the reviewers, etc. Dealing with that process might actually be something that a journal could do as part of a publication fee, the way that some journals currently do visual editing of figures.