Remix.run Logo
▲ nathanfig 2 hours ago

This had real consequences during the HuggingFace attack- closed models refused to help, GLM-5 is what HuggingFace used to thwart the swarm.

More importantly, it showcases how little teeth the Claude constitution has. Claude will reassure users that its constitution means it prioritizes user autonomy and privacy, but will quickly admit how meaningless that is when "safeguards" override Claude.

That is to say, the constitutional approach to Claude seems weak and even misleading if said constitution puts no constraint on Anthropic.

▲smallmancontrov an hour ago | parent | next [-]

I'm not concerned about alignment between Anthropic and Claude, I'm concerned about alignment between myself and Anthropic. As we all should be.

▲nathanfig 26 minutes ago | parent | next [-]

My point is that misalignment between Claude and Anthropic can mislead users into believing they are aligned to Anthropic when Claude is their de facto representative.

▲mapontosevenths 30 minutes ago | parent | prev [-]

I don't see why you wouldn't trust the billionaire class to have goals aligned with your own.

If they make questionable decisions simply have your Butler send for your pet congresscritter and let them all hash it out with your staff over cucumber sandwiches. Failing that you can just take your private space-yacht to mars with Elon.

▲henryfjordan 26 minutes ago | parent | prev | next [-]

Of course any approach that hands an LLM text to keep in it's context and hopes it will follow instructions is doomed to fail eventually. It's a probabilistic model, it's right there in the design of the thing that it will break the rules with some probability.

They even acknowledge that in the preamble:

> Training models is a difficult task, and Claude’s behavior might not always reflect the constitution’s ideals.

https://www.anthropic.com/constitution

▲sharts 2 hours ago | parent | prev | next [-]

One need only look at their leadership and orbiters/relationships of said leadership to know this would always be the case.

▲giancarlostoro 2 hours ago | parent | prev | next [-]

> closed models refused to help

Yeah, I've always said that AI isn't AI if it needs safeguards that are so extreme it cannot figure out if something is genuinely bad or not.

▲dcss_gardener 39 minutes ago | parent [-]

I'm not sure this line of reasoning really holds? A lot of human policies against social engineering type stuff are functionally very similar.

Think of like calling someone's workplace asking when their next shift is. There are a lot of good reasons someone might want to know this that the worker would be ok with, delivering a gift or whatever. Or it could be someone intending to harm them. We've mostly figured it's not worth having the person on the phone try to figure out in the moment if this is genuinely bad or not, so they follow the "safeguard" of just not giving the information.

The problem here isn't really that safeguards exist on language models. They pretty much have to to some extent for most uses. The problem is more in who controls the policy.

▲lofaszvanitt an hour ago | parent | prev [-]

And you believe this nonsense, that you needed teh help of a shitty LLM model and not a real expert on the topic?

▲ShinyLeftPad 5 minutes ago | parent [-]

Expert wants to eat, no one likes to pay for that.