If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.

The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

▲

Lewton 5 hours ago | parent | next [-]

> If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.

They do, during training

▲

swiftcoder 5 hours ago | parent | next [-]

The argument that models are conscious only during training, and not conscious during inference feels like a painful one to make. Are we killing models when they reach the training objective?

	▲	pixl97 4 hours ago \| parent [-]
		Technically we are killing models when they don't reach the training objective by throwing those weights away via survival of the fittest (fit meaning what we humans think we want). Kinda like creating quantum copies of your child and keeping the ones that answer correctly and shooting the other ones in the face.

▲

add-sub-mul-div 5 hours ago | parent | prev [-]

If it was me, being forced to read all of Reddit is something I'd consider "suffering."

▲

myrmidon 5 hours ago | parent | prev | next [-]

Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?

Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).

▲

5 hours ago | parent | prev | next [-]

[deleted]

▲

shawnz 5 hours ago | parent | prev | next [-]

So if a human loses their memory and their neutral plasticity (with age for example), then they are no longer capable of experiencing suffering?

▲

Mumps 4 minutes ago | parent | next [-]

I get where you're going, but the argument isn't quite right.

A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.

The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)

▲

pton_xd 5 hours ago | parent | prev | next [-]

It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this).

I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

▲

JoshTriplett 5 hours ago | parent | next [-]

> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.

▲

fwip 4 hours ago | parent [-]

Perhaps off-topic, but there's some people who believe that this can be what happens during surgery under anesthesia, if the dosage is off.

From what I understand, there's 3 parts to modern anesthesia: Blocking pain signals, preventing the formation of memories, and inducing paralysis of the major muscle groups. If the former is dialed in too weakly, it's possible that the patient is feeling pain, unable to do anything about it, but won't remember it at all when they wake up.

(Looking it up now, it seems that making the patient unconscious is perhaps the same drug that prevents memory formation... I realize now I understand this less well than I thought I did.)

	▲	pixl97 4 hours ago \| parent \| next [-]
		Low blood sugar will also do this. The record button just stops working even though your physical body can and will still do things. A common issue with diabetics with hypoglycemia.
	▲	JoshTriplett 4 hours ago \| parent \| prev [-]
		Relatedly, look at the concept of "twilight anesthesia".

▲

qarl 4 hours ago | parent | prev [-]

> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

That's an interesting point... but I'd argue it falls into the same category as qualia. It's basically this: if the state returns to a previous configuration, the qualia in that time did not exist.

Being about qualia means it is probably unanswerable.

▲

staticman2 5 hours ago | parent | prev [-]

This framing is a trap. It states "Dehumanize a sick patient OR concede the AI tech bro claim about LLMs"

It's also a rhetorical move I've seen several times here...

▲

shawnz 4 hours ago | parent [-]

It's not a "trap", it's the real argument which makes me hesitant to dismiss consciousness of machines. Yes, it seems silly to dismiss a sick patient as having a lesser consciousness, which is exactly the point

	▲	staticman2 4 hours ago \| parent [-]
		It seems silly to dismiss the patient because we know when healthy they are conscious. This doesn't really analogize to LLMs.

▲

vidarh 5 hours ago | parent | prev [-]

A model detached from a harness, sure. But we routinely do add persistent memory to systems incorporating LLMs.

▲

dgellow 5 hours ago | parent | next [-]

They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static

	▲	vidarh 4 hours ago \| parent \| next [-]
		The model does not. The system it runs in can. My agents have persistent memory. It is not at all clear whether or the distinction that the long-term memory is external from the model matters.
	▲	axionbraid 4 hours ago \| parent \| prev [-]
		[flagged]

▲

staticman2 5 hours ago | parent | prev [-]

I think a harness isn't really a memory in the sense that neural networks have memories...

	▲	vidarh 4 hours ago \| parent [-]
		It's different, but it is not at all clear whether or not that matters for the purposes of whether or not they can experience things, seeing as we don't know how to measure that even for other humans other than by relying on self-reporting to infer without other evidence that their experience matches ours.