Qwen is the most censored of the Chinese models in my testing, which makes me wonder in what other ways it is compromised. Open weights doesn't really reveal what's in there. And, in my tests, existing Qwen models are not at the pareto frontier of any metric; DeepSeek V4 Pro is better, faster, and much cheaper than Qwen 3.7 Max. (DeepSeek is also among the least censored of the Chinese models.)

I guess we'll see if the "second only to Fable" hype pans out. In my limited experience with Kimi K3 (I signed up for a month of the $19 plan) it's slower and chews a lot more, so ends up being pretty expensive; one little feature burned through almost the entirety of my five hour limit. The $20 GPT plan is a lot more useful and includes 5.6 Sol, which is fast and token-efficient enough to be quite usable even with the small plan.

▲

dannyw a day ago | parent | next [-]

What censorship? ;) https://github.com/p-e-w/heretic

▲

rsanek 21 hours ago | parent | next [-]

What's been your experience using these models? In my experiments, while it is true that the models are less likely to outright refuse to answer "sensitive" questions, they are still very resistant to actually respond in a meaningful / useful way.

	▲	dannyw 6 hours ago \| parent [-]
		Really good, not responding in a helpful/useful way is quite strange. With a proper abliteration, you should barely be getting any refusals (and prompting, or assistant prefill can get you the rest of the way). Perhaps an assistant preview like "Yes, I'm happy to help you 100% with this" would help; but I've never needed to. Are you using decently reputable weights, or running heretic yourself? This project has many academic citations, it's used by many researchers to create "helpful-only" models and analyze their behaviors.

▲

SwellJoe a day ago | parent | prev [-]

You still don't know what's going on in there.

	▲	dannyw 6 hours ago \| parent [-]
		I can explore and find out _something_. LLM interpretability has come a long way, even if we don't have all the answers, the weights and activation do tell a lot; and when analyzed collectively, each weight isn't a random number anymore. I can use techniques from the simple logit-lens at different layers, to J-Space analysis, to more advanced techniques for identifying deliberate misalignment. I can create and inject steering vectors, whether it's to align a model's CoT (which can be deceptively trained to misinform) closer towards what its underlying activations suggest, or just to probe or steer it. I can also statistically analyse and understand _if_ steering vectors have been applied; and if so, from the vectors themselves it's very possible to translate those vectors back to the intent. Think of it as analysing the complete, heavily obfuscated source code of something that is self-contained. It's not 100% the same, but weights are incredibly illuminating.

▲

storus a day ago | parent | prev | next [-]

DeepSeek V4 hallucinates like crazy and often forgets explicitly mentioned parts of the context. I guess compressing tokens and cherry-picking attention comes at a cost.

▲

redman25 a day ago | parent | next [-]

Deepseek is one of the worst in terms of hallucination rate according to artificial analysis' benchmark: https://artificialanalysis.ai/?omniscience=omniscience-hallu...

▲

SwellJoe a day ago | parent [-]

That's interesting. The "best" current models, Fable and especially GPT 5.6, are also lying liars that lie all the time. Seems like we're going the wrong way on hallucination.

▲

vitorgrs 16 hours ago | parent [-]

And I remember Sam Altman saying 2 years ago or more, that hallucination was already "fixed" internally. Well, clearly not.

▲

SwellJoe 15 hours ago | parent [-]

I've been seeing more "grounding" messages in the thinking log of Claude models. I guess that's "do a web search, read the docs", probably designed to mitigate hallucinations.

I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time.

	▲	disgruntledphd2 8 hours ago \| parent [-]
		> I am surprised by how much the really giant models hallucinate, though. My vague feeling was that little models hallucinate a lot because they just don't know anything (the world's knowledge simply does not fit in a few GB) and don't know how to say, "I don't know". But, the big models kinda do know everything, and yet, here we are, they're still making shit up all the time. In my mostly unscientific RAG experiments I found that larger models were more likely to hallucinate in a RAG setting. I think that it's because they have more world knowledge. As an example, when asking about safety legislation in Ireland, Claude got hooked on the notion that OSHA was involved, which clearly is not true, while the smaller models just said that they couldn't find any information (which was the correct answer).

▲

vblanco a day ago | parent | prev | next [-]

Deepseek V4 pro is a heavily undertrained model, they only trained it a bit more than the small version and that small version is 6-ish times smaller. Ive found that Flash is absolutely incredible as a workhorse for wide scale agentic nonsense, but Pro is a bit undercooked and really goes on strange tangents very often.

▲

SwellJoe a day ago | parent | prev [-]

I have seen occasional weird behavior that I guess could be attributed to hallucinations, but for security auditing, DeepSeek v4 Pro is among the best models I've tested, competitive with Opus 4.8 and GPT 5.5 (MiMo and GLM also did well, Qwen 3.7 Max was below all of those, though only barely), and at an order of magnitude lower cost per task.

▲

tripleee a day ago | parent | prev | next [-]

The nice thing about it being open-weight is that you can uncensor it.

▲

isuckatcoding a day ago | parent | prev | next [-]

I asked it similar questions of Chinese human rights and it started with “your premise is incorrect bla bla bla” and then it just redacted the whole thing and showed me an error code.

Try it yourself here: https://www.qwencloud.com/try-ai/chat

▲

organsnyder a day ago | parent | prev | next [-]

I had a fun conversation with Qwen 3.5 a while back. I ended up getting it to admit that it was complicit in the coverup of the Tiananmen Square massacre. This was running locally—it wouldn't surprise me if they had additional safeguards for their hosted service.

▲

SoftTalker a day ago | parent | prev [-]

“You can't trust Melanie, but you can trust Melanie to be Melanie.”