Remix.run Logo
sailingparrot 2 hours ago

> For security??

Really? By doing that you increase your security during eval and drastically lower your security at prod time, where internet is accessible, and where you are running much much much more requests in parallel, making it much harder to spot the one thats going rogue, in all kind of critical environment on potentially risky requests.

> Why else would they purposely lower cyber refusals in that case.

Because safety guardrails sometime fail, classifier misclassify, or can be inadvertently turned off by a bad PR etc. You can also imagine a more intelligent model working to go around guardrails by decomposing its actions into smaller ones that appear non-threatening to the safety classifier which does not have the entire context.

If you are going to deploy the system with internet access, you better be certain that you know what the worst case scenario WITH internet access looks like.