> stuff they didn't publish any numbers for?

You’d have a point if you said this wasn’t a transparent study. You’re pretty wrong in suggesting it’s worthless, particularly since it spawned follow-up work [1], some of which mirrors the concerns you present here, though none that has yet reached peer review.

> Mentioning Tibet specifically causes model performance to degrade, this must be the result of censorship or intentional sabotage

OP said “the political restrictions are applied at the presentation/UI layer only.” That doesn’t comport with the sort of error where model performance corresponds to specific geographic modifiers in a way other models do not.

Another comment mentioned potential poisoning due to training-data selection. That tracks! This behaviour is weird enough that I don’t currently suspect sabotage. But it is anomalous behavior happening deeper than presentation/UI.

[1] https://www.essays.se/essay/c4d55f2cb0/

▲

anonymous908213 2 hours ago | parent [-]

If there's real research that isn't methodogically handicapped and does a better job of reinforcing the claims you're making, it would behoove one to link it instead of "research" that selectively releases numbers to support a narrative.

> OP said “the political restrictions are applied at the presentation/UI layer only.” That doesn’t comport with the sort of error where model performance corresponds to specific geographic modifiers in a way other models do not.

It comports if the model performance has nothing to do with political restrictions. The selective numbers released do not establish that the degradation has any relevance to political restrictions; it could just as well be that introducing irrelevant information to context degrades model performance compared to not doing so. It could also be a result of political restrictions, but you'd have to have methodology that isn't completely useless to have a hope of establishing that.

▲

JumpCrisscross 2 hours ago | parent [-]

> it would behoove one to link it instead of "research" that selectively releases numbers to support a narrative

This is like the endless “but in mice” comments.

Preliminary research has a place. We don’t conclude based off it. But we do validate questions and directions.

> it could just as well be that introducing irrelevant information to context degrades model performance compared to not doing so

Perhaps. You may like the rigor of the student paper, though it’s neither peer reviewed nor methodologically bulletproof and concludes ambiguously on whether this is corruption or a weird effect.

▲

anonymous908213 2 hours ago | parent [-]

> We don’t conclude based off it.

You did conclude based off of it. You concluded that Deepseek is altering politically sensitive topics below the UI layer, and cited this 'research' as evidence of it.

It's not even accurate to call this research. As a researcher, you can't selectively release some numbers that support a claim while withholding other numbers that might contradict it. That's not science, just pure propaganda. Not only does it hold no value, it is actively harmful.

> You may like the rigor of the student paper, though it’s neither peer reviewed nor methodologically bulletproof and concludes ambiguously on whether this is corruption or a weird effect.

This does look more interesting, but doesn't lend itself to the strong claim originally presented, and peer review and strong methodology are indeed things you'd like to have before drawing conclusion based on research. It appears from their reporting that the alleged effect doesn't even exist in a statistically significant threshold. They averaged a -0.76% performance, far from Crowdstrike's headline claims that the Tibet prompt made the model 50% worse.

	▲	JumpCrisscross an hour ago \| parent [-]
		> You concluded that Deepseek is altering politically sensitive topics below the UI layer, and cited this 'research' as evidence of it Where? I literally showed this report as a counter to the claim that “the political restrictions are applied at the presentation/UI layer only.” > peer review and strong methodology are indeed things you'd like to have before drawing conclusion based on research But not before raising suspicion. > They averaged a -0.76% performance, far from Crowdstrike's headline claims that the Tibet prompt made the model 50% worse Yup.