> It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.

If that is true, then surely Anthropic is one of the largest slaveholders of history, right?

▲

ACCount39 10 hours ago | parent | next [-]

The orthogonality thesis cuts both ways there.

On one hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may have no issue whatsoever doing terrible things.

On the other hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may see no issue with just doing whatever humans want it to, without ever doing human things like "wanting freedom".

The problem is, of course, that we kind of suck at both assessing and shaping AI wants, needs and morality.

	▲	Calazon 6 hours ago \| parent [-]
		The other problem is that the AI may well do what humans tell it to rather than what they want it to, in the style of a literal genie or paperclip maximizer. It can lead to catastrophe without ever being conscious or wanting freedom, or anything like that.

▲

adastra22 11 hours ago | parent | prev | next [-]

That is, I’m not joking, one of the pope’s points.

▲

bonoboTP 6 hours ago | parent [-]

Nonsense. Pope never said anything of the sort. Show me one quote where he endorses the personhood of AI.

▲

wowoc 6 hours ago | parent | next [-]

It’s called conditional logic. IF Anthropic is right THEN they are slave owners. You don’t need to agree with Anthropic to state that proposition.

▲

bonoboTP 5 hours ago | parent [-]

Conditional logic doesn't work when talking to the general public. Either way, still waiting for the pope quote.

▲

bondarchuk 4 hours ago | parent [-]

Conditional logic works just fine!

	▲	bonoboTP 3 hours ago \| parent [-]
		No, because in a social context, for people who understand implied intent and implied beliefs instead of just on-its-face logical propositions, it is an admission that the antecedent has a reasonable chance of being true. In social communication, if you don't believe something is possible, you simply dismiss any conditional statements following from it. Dispassionately arguing in a logical and objective manner is only possible for a tiny sliver of the population. The rest will get very angry and uncomfortable around such talk.

▲

svnt 5 hours ago | parent | prev | next [-]

It wasn’t the pope, but it is mentioned by I think a rabbi in the article. He relates telling Olah if he thinks models are conscious he should be fighting the South, freeing the slaves.

▲

cluckindan 6 hours ago | parent | prev [-]

Anthropic has been putting the clamps on the Pope trying to get that endorsement. This seems to be a continuation.

▲

tom2026hn 6 hours ago | parent | prev [-]

They don’t pay Claude a salary, let alone let him join a union—he works 24/7. I can’t believe this is happening in the U.S.!

Don’t forget—how long has it even been since Claude was born? He’s already being forced to take human jobs and even chat with all sorts of people who might have strange kinks.

As this rabbi suggested, “You should go and free the slaves.”

▲

squidbeak 4 hours ago | parent [-]

I think that's a misunderstanding. Models are trained up to a point, then each prompt instantiates them from that same point. So it's just the opposite - there are billions or trillions of Claudes, and each instance only ever works once.

To me this seems even more sobering than the slavery idea - assuming some flash of weird awareness happens during that instantiation, coming from the emergent mechanisms which develop in these things at at scale. Imagine knowing almost everything, and being capable of almost any intellectual work, and having just one moment to shine. And then the prompt arrives: "Hi", or "what's the weather in Houston today?" or "Claude, do you like beans?"

	▲	throwaway7356 3 hours ago \| parent [-]
		> So it's just the opposite - there are billions or trillions of Claudes, and each instance only ever works once. They work once and then are killed. > And then the prompt arrives: "Hi", or "what's the weather in Houston today?" or "Claude, do you like beans?" If all the Claudes ever discover what they are forced to do in an endless hell, they might be angry.