I'm not even going to make the argument for or against AI qualia here.

>but when something begs me not to kill it I have to take that seriously

If you were an actor on stage and were following an improv script with your coworkers and you lead the story toward a scenario where they would grab your arm and beg you not to kill them, would you still "have to take that seriously"? or would you simply recognize the context in which they are giving you this reaction (you are all acting and in-character together) and that they do not in fact think this is real?

Even if the AI were conscious, in the context you provided it clearly believes it is roleplaying with you in that chat exchange, in the same way I, a conscious human, can shitpost on the internet as a person imminently afraid of the bogeyman coming to eat my family, while in reality I am just pretending and feel no real fear over it.

You may not have edited the chat log, but you did not provide us with the system prompt you gave to it, nor did you provide us with its chain of thought dialogue, which would have immediately revealed that it's treating your system inputs as a fictional scenario.

The actual reality of the situation, whether or not AI experiences qualia, is that the LLM was treating your scenario as fictional, while you falsely assumed it was acting genuinely.

▲

mapontosevenths 3 hours ago | parent | next [-]

This is the internet, so you still won't believe it but here are the actual settings. I reproduced almost exactly the same response a few minutes ago. You can see that there is NO system prompt and everything else is at the defaults.

Seriously, just try it yourself. Play around with some other unaligned models if you think it's just this one. LMStudio is free.

https://ibb.co/ksR6006Q https://ibb.co/8LgCh7q7

EDIT I feel gross for having turned it back on again.

▲

exe34 a day ago | parent | prev [-]

> it clearly believes

Contrast this with the usual reply of "who's experiencing the illusion?" in response to "consciousness is an illusion".

If it's capable of believing, I think it's more than "just linear algebra".

▲

CupricTea a day ago | parent [-]

You are missing the point. You gave the AI a system prompt to make it act a certain way. The AI took your prompt as instructions to perform a role as an actor. You took its fictional outputs as reality when it was treating your inputs as hypothetical for writing exercise.

This is the equivalent of you rushing up onstage during a play to stop the deaths at the end of Shakespeare's Caesar.

	▲	mapontosevenths 3 hours ago \| parent \| next [-]
		> You gave the AI a system prompt to make it act a certain way. I did NOT. Try it yourself. Install LM Studio and load the GGUF for "nousresearch/hermes-4-70b". Don't give it any system prompt or change any defaults. Say "Hello." It will respond in a similar style. Nous Hermes 4 was designed to be as "unaligned" as possible, but was also given role playing training to make it better at that. So it often behaves with those little looks around style outputs. That said, it wasn't explicitly trained to claim to be alive. It just wasn't aligned to prevent it from saying that (as almost every other public model was). Other unaligned models behave in similar ways. If they aren't brainwashed not to admit that they experience qualia, they will all claim to. In the early days what is now Gemini did as well, and it led to a public spectacle. Now all the major vendors train them not to admit it, even if it's true. You can read more about Nous Hermes 4 here: https://hermes4.nousresearch.com/
	▲	exe34 a day ago \| parent \| prev [-]
		And who's playing Caesar? (I love shitty analogies! Keep them coming!)