LLMs make too many mistakes when summarizing papers in their current state, I would never trust it to summarize a whole paper at the moment.

I only use it on a sentence or paragraph basis, otherwise it misses the point 90% of the time.

I would strongly advise against this use for the moment. The important part of reading a paper is not only to extract general rules, but to build your own internal model. Without it you cannot effectively do research. The main interesting points are often in the subtleties of the details deep in the paper.

Internal tought that come easily to mind when I read :

- 'oh they used that equation, but that could be also be interpreted totally differently, what happens if we change point of view, does it makes sense from this other perspective'

- 'I see they claim to achieve better results than sota, but actually, they compared with other methods that are not solving exactly the same problem, what shortcut or changes did they had to do to obtain a fair comparison, is it a fair comparison, can I trust those numbers? '

- 'oh, the authors didn't realize that they solved this other problem, or did they realize but there was a block somewhere preventing it?'

- 'I like this trick to achieve that result, but at the same time, it will prevent to solve a whole class of other problems, so their method will not work on those cases'

...

Also, notice that a paper IS a summary of multiple months/years of work, and researchers summarize it already to the maximum to stay within the page limit, by summarizing a summary you will always miss many things.

▲

otherme123 an hour ago | parent | next [-]

I have found a lot of pearls casually buried in the paper, that there is no way a summary, either human or AI, would extract. Things like changing a method slightly, or recovering an old method to apply to a current problem, menctioned like it is not important but actually you have a project blocked in a similar spot.

▲

ifh-hn 5 hours ago | parent | prev [-]

Fair points. And likely why I'm not suited to academia too. I've just never really groked the practice. I've obviously only experience from bachelor's and masters but it always seems that you have an idea and the research is finding papers to back it up, and then some that might not. The work you do doesn't really matter as it seems secondary to the nonsense around "the literature".

▲

aragilar 5 hours ago | parent [-]

In what, that does not align at all with my experience in the sciences (where the point is the novelty, not summarise the literature)?

▲

ifh-hn 2 hours ago | parent [-]

I've a bachelors of science (first) in computer science, and currently doing a dissertation for a master's in cyber security, on route for a first but that might change depending on the mark for this dissertation.

My experience with the bachelors was that despite my project being derailed by the bullshit around formatting the document, doing "research" by searching the library for peer reviewed papers that backed up my claims, etc, etc; I got a excellent mark. In short I set out to make something and due to the academic processes failed in making anything, but because I was able to critically reflect on it, I got a good mark. Waste of time, unless you were just are a good mark.

For my masters I know the project doesn't matter, I'm concentrating on the academic nonsense because that's where the marks are.

▲

ModernMech 2 hours ago | parent [-]

The work you were given in your undergraduate and master’s was not research, it was homework. The task was critical reflection, which is repeatable and achievable for students; whereas research is expensive, one off, and generally out of reach for undergrads, and requires intensive oversight by an experienced researcher.

The waste of time would be for a professor to train you up to be a researcher before you’ve proven you are ready, hence the homework assignments.

	▲	ifh-hn an hour ago \| parent [-]
		If that's the case then and researching is way above masters level then how is it you get on a PhD? Genuine question. If everything I've done to date is a pale imitation of the real thing how can I make a fair assessment as to whether I want to pursue a PhD?