Remix.run Logo
BrenBarn 6 days ago

They avoid mentioning the elephant in the room: jobs and tenure. When you can get hired for a tenure-track job based on your null-result publications, and can get tenure for your null-result publications, then people will publish null results. Until then, they won't hit the mainstream.

antithesizer 5 days ago | parent | next [-]

It's fascinating how utterly dominated science is by economics. Even truth itself needs an angle.

autoexec 5 days ago | parent | next [-]

The influence of money really does hold back scientific progress and is often specifically used to prevent some truths from being known or to reduce the confidence we have in those truths.

Obviously it takes money to do pretty much anything in our society but it does seem like it has way more influence that is necessary. Greed seems to corrupt everything, and even though we can identify areas where things can be improved nobody seems to be wiling or able to change course.

derektank 5 days ago | parent [-]

I don't see any way to get around the fact that science is expensive. Researching the boundaries of human knowledge requires access to rare or novel materials, which requires people to invest time and effort in producing them, which of course requires money. If your research requires access to vats of liquid oxygen, or high powered lasers, or ultra pure chemicals, or 500 pure bred rats, you'll need people working full time to produce those things for you, and they need to be able to put food on their table.

BrenBarn 3 days ago | parent [-]

There's plenty of scientific research being done outside the narrow physical-sciences type of fields you mention. Not all science is expensive in that way.

ytpete 2 days ago | parent | prev | next [-]

I would think a large part of it is simply the universities want to offer tenure to the most competent researchers. And it's harder to judge whether a researcher who mostly gets null results is doing a good job.

Perhaps this means it really does have to start with journal publications though. If journals value null results, peer reviewers will sharpen their ability to distinguish null but well-run experiments from ones that failed simply due to poor execution. Then employers can use published null results as a positive signal that a researcher is indeed doing good quality work.

pixl97 5 days ago | parent | prev | next [-]

I mean, science has always been in one way or another. All the 'scientists' of olde were either wealthy or given some kind of grant by those that were. Science itself won't be exempt from the freeloader problem either.

Not that I'm saying all science has to economic purposes.

Bluestein 5 days ago | parent | prev | next [-]

> Even truth itself needs an angle.

"When even truth itself needs an angle ...

... every lie looks like a viable alternative".-

youainti 5 days ago | parent | prev [-]

Imagine being an economist... You can't get away from it.

kurthr 5 days ago | parent | prev | next [-]

There are a few different realities here. First, it's not really whether you can get tenure with the publications, because almost none of the major respected journals accept simple null/negative results for publication. It's too "boring". Now, they do occasionally publish "surprising" null/negative results, but that's usually do to rivalry or scandal.

The counter example to some extent is medical/drug control trials, but those are pharma driven, and gov published though an academic could be on the paper, and it might find its way onto a tenure review.

Second, in the beginning there is funding. If you don't have a grant for it, you don't do the research. Most grants are for "discoveries" and those only come about from "positive" scientific results. So the first path to this is to pay people to run the experiments (that nobody wants to see "fail"). Then, you have to trust that the people running them don't screw up the actual experiment, because there are an almost infinite number of ways to do things wrong, and only experts can even make things work at all for difficult modern science. Then you hope that the statistics are done well and not skewed, and hope a major journal publishes.

Third, could a Journal of Negative Results that only published well run experiments, by respected experts, with good statistics and minimal bias be profitable? I don't know, a few exist, but I think it would take government or charity to get it off the ground, and a few big names to get people reading it for prestige. Otherwise, we're just talking about something on par with arXiv.org. It can't just be a journal that publishes every negative result or somehow reviewers have to experts in everything, since properly reviewing negative results from many fields is a HUGE challenge.

My experience writing, and getting grants/research funded, is that there's a lot of bootstrapping where you use some initial funding to do research on some interesting topics and get some initial results, before you then propose to "do" that research (which you have high confidence will succeed) so that you can get funding to finish the next phase of research (and confirm the original work) to get the next grant. It's a cycle, and you don't dare break it, because if you "fail" to get "good" results from your research, and you don't get published, then your proposals for the next set of grants will be viewed very negatively!

throwawaymaths 5 days ago | parent | prev | next [-]

This. And the incentives can be even more perverse: If you find a null result you might not want to let your competitors know, because they'll get stuck in the same sand trap.

MITSardine 5 days ago | parent | prev [-]

If people are so interested, they'd presumably read and cite null-result publications, and their authors would get the same boons as if having published a positive result.

There's some issues, though. Firstly, how do you enforce citing negative results? In the case of positive results, reviewers can ask that work be cited if it had already introduced things present in the article. This is because a publication is a claim to originality.

But how do you define originality in not following a given approach? Anyone can not have the idea of doing something. You can't well cite all the paths not followed in your work, considering you might not even be aware of a negative result publication regarding these ideas you discarded or didn't have. Bibliography is time consuming enough as it is, without having to also cite all things irrelevant.

Another issue is that the effort to write an article and get it published and, on the other side, to review it, makes it hard to justify publishing negative results. I'd say an issue is rather that many positive results are already not getting published... There's a lot of informal knowledge, as people don't have time to write 100 page papers with all the tricks and details regularly, nor reviewers to read them.

Also, I could see a larger acceptance of negative result publications bringing perverse incentives. Currently, you have to get somewhere eventually. If negative results become legitimate publications, what would e.g. PhD theses become? Oh, we tried to reinvent everything but nothing worked, here's 200 pages of negative results no-one would have reasonably tried anyways. While the current state of affairs favours incremental research, I think that is still better than no serious research at all.

michaelt 5 days ago | parent [-]

> If people are so interested, they'd presumably read and cite null-result publications,

The thing is, people mostly cite work they're building upon, and it's often difficult to build much on a null result.

If I'm an old-timey scientist trying to invent the first lightbulb, and I try a brass filament and it doesn't work, then I try a steel filament and it doesn't work, then I try an aluminium filament and it doesn't work - will anyone be interested in that?

On the other hand, if I tested platinum or carbonised paper or something else that actually works? Well, there's a lot more to build on there, and a lot more to be interested in.

directevolve 4 days ago | parent | next [-]

It depends. If I was also trying to invent a lightbulb, or maybe develop new materials for you to try as filament, I might be very interested to know what you’ve already tried.

mitthrowaway2 5 days ago | parent | prev [-]

I think the failed efforts merit at least a mention, if not a whole publication, otherwise everyone will wonder if they can save money on platinum filaments by switching to aluminum. All the lightbulb companies will continuously be re-confirming that null result internally as part of various cost-reduction R&D efforts.