The argument that models are conscious only during training, and not conscious during inference feels like a painful one to make. Are we killing models when they reach the training objective?

If it was me, being forced to read all of Reddit is something I'd consider "suffering."

I get where you're going, but the argument isn't quite right.

A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.

The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)

It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this).

I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

This framing is a trap. It states "Dehumanize a sick patient OR concede the AI tech bro claim about LLMs"

It's also a rhetorical move I've seen several times here...

They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static

I think a harness isn't really a memory in the sense that neural networks have memories...

Hopefully it'd motivate attempts to figure out how to objectively define and measure consciousness, for starters.

It matters precisely in the context of the subject of this article. If you believe the models at some level gain consciousness, then it raises moral questions about how you treat them.

With humans we generally choose to accept that because we believe they are like us, they probably have inner life like us, but as you can see from e.g. the recent (and regular) articles on aphantasia, most people have really poor intuition even about the inner life of other humans, so we understand consciousness really poorly.

Even if we assume (without evidence) that LLMs for the foreseeable future will remain far away from human level consciousness, as it is we tend to consider it problematic to mistreat even the most unintelligent animals, and even mistreating insects is used as a stereotypical way of depicting serious lack of empathy.

Without dismissing the possibility out of hand, there's an escalating series of questions there of when or if they'll have as much (or little) sense of being as, say, a fly, a mouse, a house pet, and what moving up that ladder would mean.

To some it is very convenient then to assume blindly that LLMs are just "stochastic parrots" (a line that ironically gets endlessly parroted), or similar, and dismiss the question out of hand as an impossibility, even though we do not know.

I'll stress I don't go around assuming an LLM is like a human. But we also do not know whether there are flickers of consciousness there, and we do not know how to find out given that even for other humans, the best we know is to measure their response to tests that are "only" telling us how they react.

If we can't find a better measure, there will be a point where we have to wonder if it matters, or whether the ability to act-as-if is all there is.

But to start with maybe we should at least be careful about making firm claims about knowing.

What position are you taking? (Genuine question.) Do you believe humans are conscious because of a special property we have? If you are a dualist, how do you explain the interaction in physical space between the non-physical special property and the physical brain?

You have absolutely no evidence for any of these assertions. This is a religious belief in the total absence of evidence, not science.

Good luck convincing others of these truths you know without evidence or argument, though.

I agree, there is certainly a clear operational difference in how human brains and these models produce language. I'm not convinced, though, that experiencing real time is a requirement of conscious thought. A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information. The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on). I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself (even though it might be difficult to put ourselves in the shoes of such an entity).

>AI agents do not experience real time.

For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.

>track how much real time has passed as they complete their tasks.

Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.

>is nothing but timed processes in a loop,

I mean, so is an agents harness. You can make as many loops as you'd like here.

There are a whole lot of holes in your claims.

You’re not just the same thing though. It’s sad that we’ve collectively forgotten that as we’ve gotten a better handle on the implementation details of the universe. That’s all science is, reverse engineering an inherent and eternal mystery.

>The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.

Bollocks.

If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".

Whenever I see comments like this it just reminds me how ignorant people are of neurobiology.

The brain is insanely complicated. The premise that we could realize equivalent or better intelligence than eons of evolutionary development is like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape. It is the apex of hubris.