In my opinion, the big issue with that argument is that advances in interpretability research and steering conceivably could, and probably will, render moot that (as of now, purely hypothetical) risk of subtle sabotage for open-weight models... but not for closed models.

▲

_factor 13 hours ago | parent [-]

It’s not hypothetical. Magic strings are a known and implemented feature for standard model interaction. Nearly impossible to detect unless you know where to look with current technology.

▲

wyrdcurt 11 hours ago | parent | next [-]

Maybe I should clarify. As I understand it, the kind of vulnerability being discussed is something like a Chinese model invisibly "realizing" that it's working on an American project, and then deliberately leaving subtle security bugs in its generated code for Chinese hackers to later exploit. As far as I know, that scenario is hypothetically possible, but has never been demonstrated to happen in the wild. Admittedly, I could be wrong about that! If anyone has evidence to the contrary, I'd love to see it.

Of course, one could retort that gathering that evidence may be nearly impossible now, but my point stands: in the future it might/probably will be possible to properly audit open-weight models. Closed models, on the other hand, will always be a black box.

▲

utilize1808 4 hours ago | parent [-]

They can just favour some specific versions of some library that's been compromised. Unlike introducing bugs / flaws directly in the source code, they can claim plausible deniability, and it's much easier to implement without compromising the general coding capabilities of the models.

	▲	codedokode an hour ago \| parent [-]
		Cannot non-Chinese closed weights model do the same?

▲

CamperBob2 12 hours ago | parent | prev [-]

As long as I can say, "Model A, look for security holes in this code by Model B," I don't see this being a serious problem.

It's when the vendors and/or governments in charge of Model A decide that I'm not allowed to do that, that I have a problem.