| ▲ | DonaldPShimoda an hour ago | |
I think the AI portion is not just something that ought have a toggle, but it should not be part of the platform. Somewhat recently, the ACM (one of the premier publishers for computer science) integrated AI-generated summaries for all papers, and it made these summaries appear in place of author-written abstracts; to find the abstract, users had to use a toggle. The ACM argued that this was a benefit. After significant community pushback, the ACM has swapped things: author-written abstracts now appear first, but users are still offered a toggle to access AI-generated summaries instead. As highlighted by professor Anil Madhavapeddy [1], the AI summaries are often factually incorrect, sometimes obviously, but often subtly. This sentiment was corroborated by numerous colleagues of mine less publicly: they checked the AI-generated summaries of their own papers, and for almost every paper were able to identify at least one factually incorrect or significantly misleading statement. Some people argue that AI-generated summaries help to democratize academia; I think instead they are democratizing misunderstanding. The models fundamentally lack the capacity to "understand" when what they say is wrong or misleading. It is not uncommon that I have students in office hours with severe misgivings about our course material because they asked an LLM some innocuous question to which they thought surely the LLM would generate an accurate response. The course material is, of course, drawn from various sources, so the LLM ought be fairly likely to generate accurate responses. In contrast, a publication is often (or, by definition in my field, necessarily) introducing novel conclusions; this means that the LLM is less likely to generate an accurate summary for a paper than for course materials, and the course material summaries are already problematic enough, so I think applying this to research is just a bad move. I understand the appeal. I understand how liberating it must feel to someone to get to "talk to" a paper to seek greater understanding. But if you already don't know enough about the material that this is useful, you also don't know enough to know when the responses are subtly incorrect, and I think this completely undermines the purpose of publication in the first place. | ||