The behavioral issue I see is that LLM users tend to immediately reach for an LLM and do their thinking in concert with it.

This tempts users to approach problems by first feeding them into the LLM and then simply following the route the LLM lays out, which does improve task completion time for tasks that the LLM can simply regurgitate, but it stops the user from developing the actual critical thinking skills that school is supposed to teach.

It’s not just critical thinking skills, it’s also that there’s a big difference between recognition/following instructions, and recall/generating your own memories of an approach. But most students don’t recognize the difference. In other words, “following the route” is a big part of the problem - it doesn’t engage the brain the same and isn’t representative of real world use, and having something explained well doesn’t mean you can in turn explain it well yourself (the more revealing test of internalized true understanding)

Can agree on that.

The description of the paper also said:

AI users who maintain similar homework completion time as non-AI users experience small learning losses.

This was a surprise too me. I would have thought otherwise.

Would love to see some evidence about if more or less people fall behind and have worse results. In my head the AI should be able to get the weakest students a bit highere.