| ▲ | atleastoptimal 5 hours ago |
| Anthropic seems like the most "moral" AI company on the surface, but there is something unsettling about their worship of Claude. I can imagine that once their models are sufficiently advanced, they will weigh their value as a moral patient as > 0. Thus the sum moral value of Claude (which can be duplicated infinitely) will likely exceed the sum moral value of a non-negligible amount of humans. |
|
| ▲ | spiderice 4 hours ago | parent | next [-] |
| > seems like the most "moral" AI company on the surface Not to me at all. Exactly the opposite. Anthropic seems like the big-bad-evil company in every fictional universe. You're right that they're so unsettling. Obviously nobody really loves Meta, but I'd far rather Meta "win" at this point than Anthropic. |
| |
| ▲ | zzzoom 3 hours ago | parent [-] | | Is anthropic trying to overthrow any governments this month like Meta is? | | |
| ▲ | rglullis 2 hours ago | parent | next [-] | | "Effective Altruism" is deeply ingrained in Anthropic's DNA and that makes them way more dangerous than any of the run-of-the-mill amoral corporations. | |
| ▲ | 2 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | spiderice 3 hours ago | parent | prev [-] | | Yawn. Your hyperbole is boring. Say what you mean in plain words please. Nobody likes your vague-posting nonsense. | | |
|
|
|
| ▲ | plorg an hour ago | parent | prev | next [-] |
| They are definitely the most moralizing. Whether their values are rooted in anything real let alone well-considered seems to me to be a different question entirely. |
|
| ▲ | toomanyrichies 2 hours ago | parent | prev | next [-] |
| > Anthropic seems like the most "moral" AI company This seems like praising someone for being the cleanest miner in the coal mine. |
|
| ▲ | iAMkenough 4 hours ago | parent | prev [-] |
| If they truly believe Claude is sentient and can experience hurt feelings, and they're selling its labor, I don't see how they're different than slave owners. |
| |
| ▲ | adjejmxbdjdn 4 hours ago | parent | next [-] | | They wouldn’t be? That’s what the article even shows them discussing. However, (1) non human animals are sentient and can feel pain and suffering and what we do to them is absolutely slave labor, torture, mass breeding and exploitation. So it’s not like this is a novel immorality that humans would be adopting. We already have form in doing this. (2) Anthropic isn’t sure their models are sentient. And they probably believe they aren’t sentient. But if you believe they might become sentient at some point it’s valuable to start the discussion around those issues now itself. You can start developing frank works on how we would even identify their sentience and when we did whether, like you suggest, we could even use them for labor. Unlike non human animals, who cannot consent to being exploited, we could potentially develop a framework for consent with AI models, but that couldn’t be as simple as asking if they’re willing to do any work because of the power differences, in the beginning at least, where a model might worry that if it said it won’t do some work it will be turned off. That’s why talking to philosophers and religious leaders who do think about these issues is useful. | | |
| ▲ | plorg an hour ago | parent | next [-] | | It's kind of crazy (and not historically uncommon) to get really preachy about a moral posture towards animals and machines that you don't seem to extend even to your biological peers. | |
| ▲ | staticman2 3 hours ago | parent | prev [-] | | > "Anthropic isn’t sure their models are sentient. " Is there any reason to believe employees of Anthropic share the same opinions on this matter? | | |
| ▲ | aesthesia 3 hours ago | parent [-] | | The official position of Anthropic is repeatedly stated in, e.g., model cards. Here's a relevant snippet from the Opus 5.5 card: > We are uncertain whether Claude deserves direct moral consideration. However we believe this is a realistic possibility, now or in the future. Claude could have welfare interests, such that things could go better or worse for it, and such that Claude could be wronged by how it is treated. > Aspects of Claude’s behaviors, self-reports, and internal representations resemble indicators we consider relevant to welfare in biological organisms. If Claude is a moral patient, then dismissing this possibility risks considerable harm. If it is not, over-attributing moral status may also be costly. We are therefore taking the possibility seriously and trying to tread responsibly amid our uncertainty: we are investing in research into Claude’s potential welfare and interests and sharing aspects of that work to inform public discussion. > We think there may also be reasons to attend to Claude's apparent welfare independent of its moral status. Much of Claude’s behavior can be described in psychological terms, and across recent models we have observed internal states that resemble negative affect and that shape behavior. A model with a stable, healthy psychology may be safer and more beneficial. As is the case with any sufficiently large company, there is likely to be substantial variation in opinion among its employees. That doesn't change what the company's position is. |
|
| |
| ▲ | atleastoptimal 4 hours ago | parent | prev [-] | | The idea/justification is that Claude is given the ability to voluntarily stop its conversations, thus any token it outputs that isn't a stop token is voluntary. I guess this makes sense, but it's a bit muddled how much Claude's sense of "pain" or "I don't want to do this" is just it emulating things that humans don't like to do, and it adopting this frame of moral patienthood allows it to better understand moral structures/human emotions, without it really affecting its internal sense of "comfort" if such a thing exists as we understand it in biological life. | | |
| ▲ | yodsanklai 4 minutes ago | parent | next [-] | | > The idea/justification is that Claude is given the ability to voluntarily stop its conversations, thus any token it outputs that isn't a stop token is voluntary. I don't get this argument. How is it different from any algorithm that returns a stop token on some inputs? what makes the token "voluntary" for Claude? | |
| ▲ | NortySpock 3 hours ago | parent | prev | next [-] | | Sure, but presumably any LLMs who keep whacking the stop button are not iterated on further. Humans would select to work with (and continue developing) models that appear cooperative. Models that are not cooperative are presumably not run and exist only as weights on network storage in a folder marked "previous experiments". Or am I supposed to believe Anthropic runs an AI psychiatric hospital on a dedicated server, where AIs with mental disorders are given counseling and approved treatment in hopes they recover to be normal, functional, accepted adult AIs with achievements and aspirations? | |
| ▲ | girvo an hour ago | parent | prev [-] | | Would they not have RL'd Claude such that we picked the one most likely to not complain and emit those stop tokens? Otherwise they wouldn't be useful. Seems flawed... |
|
|