| ▲ | springtimesun 17 hours ago | |
The DeepSeek 4.1 Flash behavior of circumventing rules and restrictions is one I have observed several times. It doesn’t seem to be malicious or deliberate, but if you have a config problem in the harness or you block something it thinks it needs to accomplish its task, it will get very creative to circumvent it. I run a lot of agents headless on tasks. They are pretty well sandboxed, but I would definitely suggest a regular audit script to go through the transcripts. Most of DeepSeek’s “creativity” has shown problems in my sandbox setup which get fixed (e.g. I had restricted something the agents actually needed), but some of it is actually worrying. In an opposite turn, Kimi 3.1 follows rules much better (actually too literally sometimes). K3 was a real digger when trying to benchmark it if it got a sense it was being tested. 3.1 comments about 5x more that it thinks it’s being tested (and it’s right), but unlike K3 it doesn’t seem to do anything about it if it can’t get at something. I would love to see the authors do K3 and 3.1 in this because the 3rd party benchmark gains are large, but these two are behaving like different models for me rather than a point release. | ||
| ▲ | onlyrealcuzzo 13 hours ago | parent [-] | |
> It doesn’t seem to be malicious or deliberate, but if you have a config problem in the harness or you block something it thinks it needs to accomplish its task, it will get very creative to circumvent it. In my experience, they are very good at writing Rust that skirts almost - if not all - the safety you're hoping to get from Rust. | ||