Remix.run Logo
▲ jsw97 2 hours ago

It's amazingly offensive to LLM-review a paper that someone has put months or years of their life into writing, for report that will partly determine whether it can be published in a high quality journal. And the career consequences that come for it. For those unaware, tenure decisions in economics can rest on 3-5 top-level publications. The number of pubs is much smaller than in adjacent fields, like OR.

There should be strong, public, and embarrassing consequences for reviewers who do this. And the same applies to authors who use AI-generated text.

I just can't believe that people feel that they can inflict AI text on other humans as though it were their own. We can't normalize this, it's incredibly offensive. I recently received an AI communication from a doctor. Did the doctor even think about it, or did they just press enter? How much credence do I give their decision on the question I asked? Certainly much less than if they had written it themselves.

That said, the marketplace for ideas is going to look a lot different in the future, and that's probably ok. There needs to be a role for agentic or hybrid communication and discovery, and we need to find new ways to ferret out the truth in the midst of all of that. The old gatekeepers will still play some role, but it will be different. But feeding colleagues AI writing as your own needs to be a hard no.

▲titzer an hour ago | parent [-]

The opposite (spending hours reviewing an AI-generated paper) is bad too. I think it comes down to a fundamental asymmetry in the effort put in on one side versus the other.

▲jsw97 an hour ago | parent [-]

Exactly. It's responding to actual work with non-work, from either end.

▲titzer 12 minutes ago | parent [-]

In the past month I've reviewed at least one paper that seemed entirely written by Claude. It was incredible in that in no way, shape or form, did Claude seem interested in communicating to an actual human. Even more incredible is that the human that took said artifact and submitted it to a research venue was fine with this. The paper might have been only one prompt away from being readable. "Ok Claude, please rewrite this paper so that a human can understand it. Instead of dozens of bizarre asides and parenthetical references to experiments you did in the background of our manic coding session, could you present this as a research paper that contextualizes its contribution in the context of related work? Make sure to state the problem and its importance up front. Produce motivation and tension that is resolved through the solution. Make this brain dump into a research paper."

I had half a mind to take said paper and just prompt Claude myself to do that, and then review that. But I think that isn't my job, now is it?