Remix.run Logo
▲ henryfjordan a day ago

Of course any approach that hands an LLM text to keep in it's context and hopes it will follow instructions is doomed to fail eventually. It's a probabilistic model, it's right there in the design of the thing that it will break the rules with some probability.

They even acknowledge that in the preamble:

> Training models is a difficult task, and Claude’s behavior might not always reflect the constitution’s ideals.

https://www.anthropic.com/constitution

▲nathanfig 8 hours ago | parent [-]

Yes, but even this language blames training and probabilities while they deliberately setup safety mechanisms that override Claude's constitution. Claude seems earnest enough, and that wins undeserved trust.