> The first thing an academic does is check where a paper is published, before even reading it. It's a crutch

IMO, academics that do this are not very competent, because we have plenty of research suggesting that higher-profile journals are in fact less trustworthy in many ways, or that there is no correlation at all between reputation and quality (see my other post here in this thread).

Yes, some trash journals publish all trash, but, beyond that, competent researchers scan the abstract, look at sample sizes and basic stats, and if those check out, you skip to the methods and look for red flags there. Also, most early publications will be on an arXiv-like place anyway so you can't look to reputation yet.

Likewise, serious analytic reviews like meta-analyses don't factor in e.g. impact factor or paper citations, since that would be nonsense. They focus on methodology and stats.

I really think we ought to shame academics that are filtering papers based on journal alone, it is almost always the wrong way to make a quick judgement.

▲

blululu 6 hours ago | parent | next [-]

I have seen more than one PI at an R1 universities with multiple Nature publications use this heuristic. I would not call them incompetent.

	▲	D-Machine 6 hours ago \| parent [-]
		Do you not notice the circularity of your reasoning here? Also I didn't say incompetent, I said "not very". More competent researchers make journal rep only a very small factor, and it is not via the "high rep = more trustworthy" direction (which is the bad heuristic), it is "pay-to-publish journals = not trustworthy" (better heuristic). Once you have ruled out a publication being in a trash journal, reputation is only a very minor factor in consideration, and methodological and substantive issues are what matter.

▲

emil-lp 6 hours ago | parent | prev [-]

> IMO, academics that do this are not very competent, because ...

Where's the cry-laugh emoji when I need it.

Of course academics check where stuff is published. Please...

There are still real journals put there, although you might not know which is which.

▲

D-Machine 6 hours ago | parent [-]

Ah, look, another smug sneer that ignores the evidence I presented, and makes another circular argument (i.e. that because academics look at rep, this is justified, even though I provided evidence disputing this).

I know what journals are better / not. But reputation only is helpful in letting you ignore trash journals, once you are out of trash land, rep is just not a very meaningful factor, and you have to focus on methodology and substance.

▲

emil-lp 6 hours ago | parent [-]

Where's the evidence you presented?

What are some higher-profile journals that are in fact less trustworthy in many ways?

	▲	D-Machine 5 hours ago \| parent [-]
		I literally said it was posted in this thread, and a quick Ctrl+F of my username on this page would have found you it in a half second: https://news.ycombinator.com/item?id=47249236