Remix.run Logo
▲ unethical_ban an hour ago

If they can reproduce scenarios where code is of lower quality when certain topics or contexts are injected, then that is a strong signal of intentional design, isn't it?

▲jazzyjackson an hour ago | parent [-]

No; it’s a signal of lobotomy. Models get dumber near where you sensor them. You could almost think of it like your prompt is getting redacted, and so the LLM has to deal with these holes in your prompt (where you mentioned the semantic contraband) and it doesn’t do a very good job because it has no idea what you’re talking about.

The mechanism is likely a bit inverted, it’s the weights that have been redacted, but the effect is that weights that were interfered with are affecting the problem solving abilities of the LLM - I mean, it’s insecure bad code, not a backdoor, right?