| |
| ▲ | orbital-decay 19 minutes ago | parent | next [-] | | Intelligence has little to do with it, Opus relies on prompt injections done by a pretty zealous classifier or a cascade of classifiers. It has a ton of semi-black box machinery around it, which can be taken apart. Despite Anthropic's benchmarks and claims, Claude was always the easiest to deceive or JB if you know how to do it compared to e.g. GPT, which the RP community knows pretty well. They definitely improved it in 5.5 (due to distillation fears, ironically), but I'm not trusting their or anyone else's numbers this time either. These benchmarks always look good, and then there's someone routinely jailbreaking it layer through layer just for fun. | |
| ▲ | modeless an hour ago | parent | prev [-] | | Still, all instances of Opus are identical and do not learn. As long as the rate of mistakes is nonzero, my point stands. | | |
| ▲ | himata4113 an hour ago | parent | next [-] | | I believe that still reduces the workload by a huge factor as you would need a lot less people handling approvals, besides inverse is true agents trained in detecting deception might outperform human verifiers (two agent system). If a model can hold up to attacks for a long period of time the approval process can be automated as well, but this would have to be a very slow rollout spanning more than a year. | |
| ▲ | tyre 31 minutes ago | parent | prev [-] | | Not really. You can still put limits on the rate or aggregate dollar amount of refunds or transfers. You can put alerting on spikes of approvals. You can put back pressure that forces manual review. You can have ML models detecting aberrant behavior against historical data. No one reasonable is saying, “hook up this model to do anything it wants with no other system available.” Will people fuck it up? Absolutely! But, “the model has to be perfect or it will bankrupt a company” is more cynical than reality. |
|
|
