Remix.run Logo
▲ nneonneo 5 hours ago

1. Were these the only prompts tested, or are these selected comparisons? I note that prior work used other triggers, like Tibet, which are omitted here - were they not tested, or did they simply fail to show similar results?

2. This is an old model that isn’t very good at writing secure code in the first place; I’m not sure it’s an improvement to use a “westernized” model that still writes insecure code 50+% of the time. If this is a genuine issue, it should be demonstrable with later models and with models that are more secure-by-default.