Isn't this just showing the effects of actively engaging the learner by placing a topic in contexts familiar/favored by learner versus just reading about a topic?

Like, if you had made the text pdf readers do some manual thinking by working on trying to place the topic into the same type of familiar/favored context, wouldn't that have been the better comparison?

I think using GenAI for learning is cool and exciting (especially for autodidacts) but I'm not excited by this particular study structure.

▲

non_aligned 4 days ago | parent [-]

> I think using GenAI for learning is cool and exciting (especially for autodidacts)

I don't know. I've been trying, but I think there are two fundamental issues. First, I don't think it's all that useful for "out-of-order" learning and for explaining concepts in non-conventional ways.

To give you a practical example, there's a certain order in which we teach math, and every subsequent step builds on the previous one. But quite often, this order is just a matter of convention, not necessity. You can explain a ton of higher-math concepts in terms of high-school algebra and geometry, it's just not something we do because we don't intend to teach high-schoolers any of that, and for undergrads, it's more expedient to lean on their knowledge of calculus / mathematical analysis than to start by drawing triangles.

And not once have I succeeded in convincing an LLM to circumvent that. If a topic is explained using mathematical analysis in college textbooks, this is how it will always answer. Which actually sucks for that curious high-schooler.

But second, LLMs just aren't nearly as dependable as textbooks. It's not even the base error rate - I think they're 90%+ accurate on most run-of-the-mill scientific questions - but that once they make a confidently-sounding mistake and you try to drill down, they keep digging that hole and sending you more and more off track. It's amusing if you know the domain and can spot mistakes. It's a huge waste of time when trying to learn a new field.

It's precisely why vibe-coding is more useful to experienced developers who can immediately reject bad results.

	▲	clusterhacks 4 days ago \| parent [-]
		"Out-of-order" and non-conventional explanations are interesting to consider. For both, I would expect LLMs to do poorly when there isn't much (or any) material for those approaches in the training data. My intuition would be the learner is going to have to be more exploratory via prompt engineering and still struggle against the tendency of the model to lean into classic or conventional explanations. I don't particularly expect models to be dependable in responses, but I see how that presents a much larger problem in a learning context. I'm ok with bad responses that I can fight back against, but I also wouldn't reach for an LLM for a new field by default either. For me, I do like using an LLM as a supplemental learning aid along with other traditional resources. I haven't tackled a deeper, new-to-me field yet with one. Maybe it's time for that . . .