| ▲ | tekacs 9 hours ago | |
I really hope that one day we reach a point where we feel confident enough in the standards of care in upstream software, that we can get rid of these safeguards. This isn't said out of naiveté or the idea that companies won't cheap out, but at some point – if access to models for defense is broadly available enough – we have to take a step back and say, "Aggressively insure your code against attack with AI on your side, because after <date> the other side will just _have_ AI." I feel like something lost in a lot of the discussion around mythos and fable is that computer security absolutely has a substantial defender's advantage. It is indeed possible to ship e.g. surfaces that would be super resilient to attack (e.g. no unnecessary open surfaces, etc.) modulo category-shift attacks like RowHammer, etc. Besides, just making sure that more people in the world actually have access to non-lobotomized models, this is _necessary_ if open source is (hopefully) going to continue to progress and if jailbreaks aren't totally vanquished. | ||