I skimmed the paper and I question the validity of the experiment.

There was a “brain” group who did three sessions of essay writing and on the fourth session, they used ChatGPT. The paper’s authors said during the fourth session, the brain groups EEG was higher than the LLM groups EEG when they also used ChatGPT.

I interpret this as the brain group did things the hard way and when they did things the easy way, their brains were still expecting the same cognitive load.

But isn’t the point of writing an essay is the quality of the essay? The LLM supposedly brain damaged group still produced an essay for session 4 that was graded “high” by both AI and human judges but were faulted for “stood out less” in terms of distance in n-gram usage compared to the other groups? I think this making a mountain out of a very small mole hill.

▲

MobiusHorizons 4 days ago | parent [-]

> But isn’t the point of writing an essay is the quality of the essay

Most of the things you write in an educational context are about learning, not about producing something of value. Productivity in a learning context is usually the wrong lens. The same thing is true IMO for learning on the job, where it is typically expected that productivity will initially be low while experience is low, but should increase over time.

	▲	rawgabbit 4 days ago \| parent [-]
		That may be true if this was measuring an English class. But the experiment was just writing essays, there were no other instruction other write an essay either with no tools, ChatGPT, or a search engine. That is the only variable was tool or without tool.