From the judge prompt in the paper:

> Papers asking whether LLMs have such properties are assuming them (e.g., ‘Do LLMs have musical talent’, ‘Do LLMs present empathy’, etc).

This seems like...a very bad definition of "assuming" something? If I ask "do you know how to play the guitar?" I am absolutely not assuming that you know how to play the guitar!

▲

avianlyric 6 hours ago | parent | next [-]

Isn’t the entire paper is trying to point out that the second you ask the question “Do LLM have <anthropomorphic property X>”, you have to assume that they do, even before you make any assessment?

Just because the person asking the question isn’t aware of they’re implicitly making that assumption, doesn’t change the fact that a logical assumption has been made. It just makes the questioner ignorant of the assumptions they’re making.

Personally don’t totally understand the argument being made in the paper. But I can understand the idea that I can ask a question, without properly understanding the assumptions I’m making when asking the questions. Indeed I can also understand that I might not even notice the assumptions I’ve made with my question, and why that would make my entire exploration and conclusion invalid, _after_ doing the investigation. Logical fallacies can be really difficult to spot and understand.

	▲	NooneAtAll3 an hour ago \| parent [-]
		> the second you ask the question “Do <things> have <property>”, you have to assume that they do being able to imagine something doesn't mean believing in it? I completely fail to understand the argument I feel like there's some mistake in confusing 2 meanings of "assume" - one where it's close to 100% probability and one where it's close to 0% probability.

▲

sublinear 4 hours ago | parent | prev [-]

You're still assuming the person is capable of playing the guitar.

Does your fridge play the banjo? Doesn't make sense does it?

▲

aesthesia 2 hours ago | parent | next [-]

Yes, asking the question does assumes that the answer could be yes. It also assumes that the answer could be no. This is exactly the kind of scientific approach the paper claims we should take. So it's certainly a bit odd that the analysis approach the paper uses for its literature review---to claim that 57% of papers reviewed assume LLMs have anthropomorphic attributes---has "asks whether an LLM has an anthropomorphic attribute" as one of its criteria for concluding "assumes LLMs have anthropomorphic attributes."

▲

golly_ned an hour ago | parent [-]

You're right that "assumes" might be misleading. Maybe "implies" would be more correct.

The point the author's trying to make is that if we state in a paper something like "the LLM understands, believes, thinks, ..." then we're supposing an intelligence much like our conception of a human intelligence. It's a form of 'begging the question' -- assuming what you're trying to prove.

It is not quite a fair argument, just because we don't have a precise vocabulary around how to talk about the activity of LLMs that doesn't involve making these loose analogies. Except for philosophers and people engaging in this kind of "is it truly intelligent or no" conversation, being imprecise in this way doesn't necessary have any cost, but is just a convenient way to avoid developing a jargon.

	▲	sublinear 20 minutes ago \| parent [-]
		> we don't have a precise vocabulary... We do, but much of computer science is still inaccessible to the layperson. The education gap only continues to grow. I think it's surprising how much science jargon we've been able to cram into common english thus far without losing too many people. It just seems that, for now, LLMs are too convincingly close to science fiction for people to not be misled by their false intuitions and fears.

▲

NooneAtAll3 an hour ago | parent | prev [-]

> Does your fridge play the banjo? Doesn't make sense does it?

have you ever heard DnD story about gazebo?

if you don't know anything about something, anything is possible and everything can make sense