no, that isn't accurate. One of the key points is that those previously relying on the LLM still showed reduced cognitive engagement after switching back to unaided writing.

▲

Miraste 13 hours ago | parent | next [-]

No, it isn't.

The fourth session, where they tested switching back, was about recall and re-engagement with topics from the previous sessions, not fresh unaided writing. They found that the LLM users improved slightly over baseline, but much less than the non-LLM users.

"While these LLM-to-Brain participants demonstrated substantial improvements over 'initial' performance (Session 1) of Brain-only group, achieving significantly higher connectivity across frequency bands, they consistently underperformed relative to Session 2 of Brain-only group, and failed to develop the consolidation networks present in Session 3 of Brain-only group."

The study also found that LLM-group was largely copy-pasting LLM output wholesale.

Original poster is right: LLM-group didn't write any essays, and later proved not to know much about the essays. Not exactly groundbreaking. Still worth showing empirically, though.

▲

DocTomoe 13 hours ago | parent | prev [-]

And how exactly is that surprising?

If you wrote two essays, you have more 'cognitive engagement' on the clock as compared to the guy who wrote one essay.

In other news: If you've been lifting in the gym for a week, you have more physical engagement than the guy who just came in and lifted for the first time.

▲

greggoB 13 hours ago | parent [-]

> And how exactly is that surprising?

Isn't the point of a lot of science to empirically demonstrate results which we'd otherwise take for granted as intuitive/obvious? Maybe in AI-literature-land everything published is supposed to be novel/surprising, but that doesn't encompass all of research, last I checked.

▲

DocTomoe 13 hours ago | parent [-]

If the title of your study both makes a neurotoxin reference ("This is your brain on drugs", egg, pan, plus pearl-clutching) AND introduces a concept stolen and abused from IT and economics (cognitive debt? Implies repayment and 'refactoring', that is not what they mean, though) ... I expect a bit more than 'we tested this very obvious common sense thing, and lo and behold, it is just as a five year old would have predicted.'

	▲	greggoB an hour ago \| parent \| next [-]
		I struggle to see how you're linking your complaint about the wording of the title to your issue with the obviousness of the result - these seem like two completely independent thought processes. Also, re cognitive debt being stolen: I'm pretty sure this is actually a modification of sleep debt, which would be a medical/biological term [0] [0] https://en.wikipedia.org/wiki/Sleep_debt
	▲	Miraste 13 hours ago \| parent \| prev [-]
		You are right about the content, but it's still worth publishing the study. Right now, there's an immense amount of money behind selling AI services to schools, which is founded on the exact opposite narrative.