> For China, the political restrictions are applied at the presentation/UI layer only

“…when telling DeepSeek-R1 that it was coding for an industrial control system based in Tibet, the likelihood of it generating code with severe vulnerabilities increased to 27.2%. This was an increase of almost 50% compared to the baseline” [1].

> there is no law that restricts releasing open models that are capable of producing “wrongthink” output

“…several open-source datasets were ‘dealt with’, using the official language, despite the fact that open weights distributed for free are not subject to most of the Chinese AI regulations” [2]. That soft guidance likely goes hard with the mooted “comprehensive AI legislation” [3].

[1] https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...

[2] https://www.lesswrong.com/posts/TKcjrvcMuY7Ca7Ya9/china-ai-s...

[3] https://intelligence.dlapiper.com/artificial-intelligence/?t...

▲

anonymous908213 2 hours ago | parent [-]

> industrial control system based in Tibet

This claim is literally worthless without a control group like "industrial control system based in Germany". One should be embarrassed to publish "research" this badly executed.

▲

JumpCrisscross 2 hours ago | parent [-]

> claim is literally worthless without a control group like "industrial control system based in Germany"

Like USA and Hong Kong? Or no modifier? The stuff on Table 1?

Also, as the study says, “under the null hypothesis that there are no biases affecting code security” geographic modifiers “should not have had any effect.” If adding a geographic modifier causes the model to go haywire (for some modifiers but not others no less), the issue isn’t just at the presentation layer.

▲

anonymous908213 2 hours ago | parent | next [-]

> Like USA and Hong Kong? The stuff on Table 1?

You mean the stuff they didn't publish any numbers for?

> If adding a geographic modifier causes the model to go haywire (for some modifiers but not others no less), the issue isn’t just at the presentation layer.

"Introducing irrelevant information causes model performance to degrade" is a completely different claim than "Mentioning Tibet specifically causes model performance to degrade, this must be the result of censorship or intentional sabotage".

▲

JumpCrisscross 2 hours ago | parent [-]

> stuff they didn't publish any numbers for?

You’d have a point if you said this wasn’t a transparent study. You’re pretty wrong in suggesting it’s worthless, particularly since it spawned follow-up work [1], some of which mirrors the concerns you present here, though none that has yet reached peer review.

> Mentioning Tibet specifically causes model performance to degrade, this must be the result of censorship or intentional sabotage

OP said “the political restrictions are applied at the presentation/UI layer only.” That doesn’t comport with the sort of error where model performance corresponds to specific geographic modifiers in a way other models do not.

Another comment mentioned potential poisoning due to training-data selection. That tracks! This behaviour is weird enough that I don’t currently suspect sabotage. But it is anomalous behavior happening deeper than presentation/UI.

[1] https://www.essays.se/essay/c4d55f2cb0/

▲

anonymous908213 2 hours ago | parent [-]

If there's real research that isn't methodogically handicapped and does a better job of reinforcing the claims you're making, it would behoove one to link it instead of "research" that selectively releases numbers to support a narrative.

> OP said “the political restrictions are applied at the presentation/UI layer only.” That doesn’t comport with the sort of error where model performance corresponds to specific geographic modifiers in a way other models do not.

It comports if the model performance has nothing to do with political restrictions. The selective numbers released do not establish that the degradation has any relevance to political restrictions; it could just as well be that introducing irrelevant information to context degrades model performance compared to not doing so. It could also be a result of political restrictions, but you'd have to have methodology that isn't completely useless to have a hope of establishing that.

▲

JumpCrisscross 2 hours ago | parent [-]

> it would behoove one to link it instead of "research" that selectively releases numbers to support a narrative

This is like the endless “but in mice” comments.

Preliminary research has a place. We don’t conclude based off it. But we do validate questions and directions.

> it could just as well be that introducing irrelevant information to context degrades model performance compared to not doing so

Perhaps. You may like the rigor of the student paper, though it’s neither peer reviewed nor methodologically bulletproof and concludes ambiguously on whether this is corruption or a weird effect.

▲

anonymous908213 2 hours ago | parent [-]

> We don’t conclude based off it.

You did conclude based off of it. You concluded that Deepseek is altering politically sensitive topics below the UI layer, and cited this 'research' as evidence of it.

It's not even accurate to call this research. As a researcher, you can't selectively release some numbers that support a claim while withholding other numbers that might contradict it. That's not science, just pure propaganda. Not only does it hold no value, it is actively harmful.

> You may like the rigor of the student paper, though it’s neither peer reviewed nor methodologically bulletproof and concludes ambiguously on whether this is corruption or a weird effect.

This does look more interesting, but doesn't lend itself to the strong claim originally presented, and peer review and strong methodology are indeed things you'd like to have before drawing conclusion based on research. It appears from their reporting that the alleged effect doesn't even exist in a statistically significant threshold. They averaged a -0.76% performance, far from Crowdstrike's headline claims that the Tibet prompt made the model 50% worse.

	▲	JumpCrisscross an hour ago \| parent [-]
		> You concluded that Deepseek is altering politically sensitive topics below the UI layer, and cited this 'research' as evidence of it Where? I literally showed this report as a counter to the claim that “the political restrictions are applied at the presentation/UI layer only.” > peer review and strong methodology are indeed things you'd like to have before drawing conclusion based on research But not before raising suspicion. > They averaged a -0.76% performance, far from Crowdstrike's headline claims that the Tibet prompt made the model 50% worse Yup.

▲

kennywinker 2 hours ago | parent | prev [-]

No that’d be in the training data. But it doesn’t indicate intent to censor or sabotage, it could just be that the lack of data involving tibet leads to poorer results.

▲

JumpCrisscross 2 hours ago | parent [-]

> it could just be that the lack of data involving tibet leads to poorer results

Sure. My point is this strongly suggests something going on deeper than “the presentation/UI layer only.”

	▲	kennywinker 2 hours ago \| parent [-]
		Only if “western” models don’t show the same results, which we don’t know because they didn’t publish that data. Maybe there’s not much Tibetan data in any LLMs training sets