Remix.run Logo
tremon 4 days ago

There is no way to answer that question, period. Neither the term "psychology" nor the term "sociology" is defined in the text (and the terms are used in union, not constrasted), so to have any hope of answering at all you need to apply knowledge not found in the text, which is expressly prohibited.

(edit to add:)

This is just regular poor-quality language model output. The language model is trained on data where the phrase "based on the provided text" makes a common appearance between a text segment and subsequent questions, but the model has no knowledge of the very specific, limiting meaning of that phrase: it limits the following question to assess reading comprehension only, not general knowledge.

So no, I don't think this is a minor bug that's easily fixable: a pure language model will always associate key signaling phrases with the wrong type questions, because it has no concept of (didactic) mode. It basically considers all phrases ornamental instead of purposeful.

bonoboTP 3 days ago | parent [-]

> assess reading comprehension only, not general knowledge.

It looks like an LSAT question. Exactly the opposite of a knowledge question. These reading comprehension questions are designed to test whether you can - without any specific knowledge of the domain area - figure out what a piece of text claims, without injecting your own knowledge or assumptions. It's an ill fit for teaching a subject matter, though can be useful for reading and argumentation (especially in lawyer-like jobs, hence LSAT).