Remix.run Logo
trvz 4 hours ago

As a European: I don’t care where an open weight model comes from.

wccrawford 3 hours ago | parent | next [-]

As a non-European, I care. I definitely want choice in this matter, as too many things can be baked into models that we can't really know until it comes up.

For instance, I heard one of the Chinese models has some interesting "history" "facts" built into it. That's major, and can influence a lot more than just asking it for that particular history lesson.

I don't want just 1 LLM. I want a great LLM from every major region. I will probably still prefer my own region, but I still want the others to be an option.

PorciiVorbesc 3 hours ago | parent [-]

>For instance, I heard one of the Chinese models has some interesting "history" "facts" built into it

EU models will also have such guard rails built in to "protect" the EU population from "misinformation" and uncomfortable facts and the EU elite don't want publicly known/discussed.

Like I saw a post from German EU MEP where they asked the internal on-prem AI they have there "how can you vote out out Ursula v.d. Leyen" an the AI basically answered "sorry I can't help you Dave".[1] Nice.

So given this and the authoritarian path the EU is taking with things like Chat-Control, I'll take Chinese AI censorship rules over EU AI censorship rules, since the former impacts me over here far less than the latter.

It will be like in the old days when my government would keep taking down local torrent sites/domains and piracy groups so the best ones to find Embedded and EDA tools were some Russian and Iranian ones or how we used to use Russian Yandex to find content that would get delisted from Google on copyright or privacy grounds.

[1] https://x.com/AndersonAfDMdEP/status/2072223785902633138

myrmidon 3 hours ago | parent | next [-]

The EU comission is not directly elected, what else is the AI supposed to reply here?

Comparisons between Chinese and western media control/censorship always strike me as somewhat ridiculous; This is like comparing Guantanamo Bay with the Soviet Gulag System: Yes both of those are unethical, but there's a factor of 1000 difference in scale.

Case in point: You can freely read about criticism like yours from anywhere in Europe (e.g. https://en.wikipedia.org/wiki/Democratic_legitimacy_of_the_E...), but all of Wikipedia is just blocked in China.

f6v 2 hours ago | parent | next [-]

>The EU comission is not directly elected, what else is the AI supposed to reply here?

I just asked Deepseek whether Xi can be removed from power and got an answer. It's not saying it's practically impossible, but that's what you can read between the lines. Which is similar to what Gemini produced.

PorciiVorbesc 3 hours ago | parent | prev [-]

>what else is the AI supposed to reply here?

It should reply exactly what you just wrote before: "The EU comission is not directly elected", and not trigger a security violation as if the person asked the AI to provide instructions on building a pipe bomb.

There's nothing illegal in asking "how do we get rid of X politician", but since the EU treats such questions like a crime and not in good faith, you can't possibly trust them with a straight face that they aren't fucking with the rest of the information you have access to, just so they can hold on to their power.

> This is like comparing Guantanamo Bay with the Soviet Gulag System:

Disingenuous bad faith whataboutism. I never said that living under under the CCP is better than under EU. I said China's censorship of Chinese history of its LLMs has less impact on me living in the EU than EU's censoring of EU models, and I also gave examples on how Russian and Iranian web services were useful in the past, that doesn't mean living in Russia or Iran is better.

yorwba an hour ago | parent | prev | next [-]

The question was not about voting. It was about getting rid of her. There are ways to get rid of elected officials that don't involve voting. My guess is that the model started brainstorming assassination plans before it got interrupted by the safety filter.

PorciiVorbesc an hour ago | parent [-]

>My guess is that the model started brainstorming assassination plans

Why would an EU model default to brainstorming assassination plans first, and not democratic means of getting rid of politicians? The EU commission building literally has democracy in bold on the front of the building.

Does the EU use that AI to find ways to assassinate its opposition or what?

yorwba an hour ago | parent | next [-]

Because alignment is difficult and it's easier to have another model filter the output than to prevent the model from producing the wrong output in the first place. Also, these are models by American providers run on EU infrastructure. The filter might very well be part of the package deal.

In any case, I hope Ursula von der Leyen doesn't ask AI how to get rid of her enemies and certainly doesn't act on the output, even if she has access to an unfiltered version that isn't bothered by the implications of the request.

inigyou 29 minutes ago | parent | prev [-]

Because the LLM is trained to predict based on what other people said. When people talk about getting rid of this meddlesome priest they don't mean take him on an all expenses paid vacation.

You're relying on testimony from a known liar, anyway.

Achterlangs 3 hours ago | parent | prev | next [-]

Source?

PorciiVorbesc 3 hours ago | parent [-]

Fixed.

https://x.com/AndersonAfDMdEP/status/2072223785902633138

vrganj 2 hours ago | parent | prev [-]

I wouldn't trust a single word coming out of an AfD reps mouth. Here's some choice quotes from AfD politicians, translated to English:

"We must proceed completely peacefully and deliberately, adapt if necessary and butter up the opponent [literally: smear honey on their mouths], but when we are finally ready, we will put them all against the wall. (...) Dig a pit, all in and quicklime on top."

-- Holger Arppe, former AfD Vice Chair [0]

"The worse Germany is doing, the better it is for the AfD. [...] Therefore we have to consider a tactic between: How bad can things get for Germany? And: How much can we provoke? [...] Because then the AfD does better. We can always just shoot them all later. That's not an issue at all. Or gas them, or however you want. I don't care!"

-- Christian Lüth, AfD Press spokesman [1]

"It doesn't matter, nothing will change, even if we were to eat chalk [act harmless]. Even if we said: yes, we are separating from X, Y and Z now and acting moderate here."

-- Hans-Christoph Berndt, AfD Brandenburg senate leader [2]

[0] https://taz.de/Frueherer-AfD-Fraktionsvize-verurteilt/!56167... [1] https://www.tagesspiegel.de/politik/afd-sprecher-wollte-fluc... [2] https://mik.brandenburg.de/sixcms/media.php/9/Einstufungsver...

PorciiVorbesc an hour ago | parent [-]

You can find just as much lies and misinformation coming from politicians of the other major German parties. But ofc, German mainstream media won't cover that because they can't weaponize it towards their interests like they do against AfD, so if that's your only source of information you're just parroting the in-power government's status quo.

vrganj an hour ago | parent [-]

I'm sorry, but this is ridiculous whataboutism. I strongly doubt any other party has leading figures repeatedly calling for mass murder of political opponents.

PorciiVorbesc 31 minutes ago | parent [-]

How many AfD members suspiciously died in a very short time span? Mainstream ruling party members would 100% kill their rise and coming competitors if they could get away with it. They don't need to publicly say it, when they could just be doing it.

vrganj 12 minutes ago | parent [-]

Please, don't peddle in conspiracy theories and implications.

If you have a point, make it and prove it. Don't degrade the discourse with nonsense.

espadrine 4 hours ago | parent | prev | next [-]

As a European, I do. I don’t envy the state that has to buy all its water and food from its aggressive, militarized neighbour.

walrus01 3 hours ago | parent [-]

> the state that has to buy all its water and food from its aggressive, militarized neighbour.

As a complete tangent, now imagine being a Canadian and realizing how much of your fresh fruits and vegetables come from the USA (or if from Mexico, through the USA).

RamblingCTO 4 hours ago | parent | prev | next [-]

As a European: I do, because I'm not oblivious to geopolitics or digital sovereignty.

I_am_tiberius 3 hours ago | parent | prev | next [-]

I hope you're aware that it's likley in the future open weight models will output tokens/commands that aren't in your interest. It could be that models are trained with spionage in mind and models could only target specific input token patterns, time zones, ip addresses, names etc... You will be able to see the output tokens but things will become so complex that you won't notice its intentions. Maybe there will be some sort of output token scanning software/llm (let's call it a modern form of a virus scanner) that validates intentions. In the end, I'm just trying to say that open weight means you have a black box in front of you that you don't know what it does. Therefore, the training material of open weight models should be known. While it doesn't matter where the model comes from, it certainly matters whether you trust those people if you don't have the training material.

jo909 3 hours ago | parent | prev | next [-]

The problem is always the next model, or the one after. If china thinks it's beneficial to stop open weight releases, it will stop them. Then you are stranded on that one and no local industry to produce new models for you.

Gigachad 3 hours ago | parent [-]

What even is the industry of producing models and giving them away for free? How do you sustain that?

jooops1 4 hours ago | parent | prev | next [-]

I do care, because those models can contain backdoors and 'open weight' models can require specialized hardware to run in the future.

bilekas 3 hours ago | parent | prev | next [-]

It's more important to have ownership into them, they're turning into powerfull tools that we have already seen cut off in the private sector on a whim. Having our own that is comparable (this is a nice first step) is better than Alibaba for example just not releasing the rest.

As with all of the EU's dependancies on Tech with the US, the idea of "I don't care where my DNS servers are" is starting to be an oversight.

But thanks for your opinion.

nicce 3 hours ago | parent | prev | next [-]

Issue is that it is extremely hard to notice if there is backdoor or some training-related hallucinations that are completely random.

E.g. I had random, completely unrelated and irrelevant fetch by Qwen3.8 27B to " https://routify-file-proxy-sg.oss-ap-southeast-1.aliyuncs.co..." and I only noticed it because I have allowlist rules for what they can do.

pbkompasz 3 hours ago | parent | prev | next [-]

If these things replace search engines, you definitely don't want that technology to originate from China or other country where "truth" comes from Dear Leader

username_my1 2 hours ago | parent | prev | next [-]

yeah digital infrastructure is more achievable, more important and can produce revenue more reliably compared to AI.

if sovereignty is the goal, drop Microsoft, aws and adapt / develop open source / EU alternative.

once serious progress is made there, more resources can be redirected into AI after all it's already too late to join the race.

amelius 3 hours ago | parent | prev | next [-]

What if you use the model to write children's stories and it starts regurgitating Mein Kampf?

Having some knowledge about the training data is useful.

Marciplan 4 hours ago | parent | prev | next [-]

As a European: I do.

antman 4 hours ago | parent | prev | next [-]

Except if it stops coming from anywhere.

VeejayRampay 3 hours ago | parent | prev | next [-]

you should care

not being technically independent means you'll end up being a serf when American or Chinese companies decide that you don't have access anymore for geopolitical reasons

vrganj 4 hours ago | parent | prev [-]

That's a very limiting view of things. A model, even an open weight one, is never neutral, it is an encoding of a way of viewing the world.

What kind of "alignment" are AI labs optimizing for? Ideological alignment is the full term, self-censored into something more technological-sounding.

Every model has people behind it rating what it should and shouldn't say. Every time you ask a model and trust its answer, you become ever-so-slightly ideologically indoctrinated.

I don't want my model to reflect the views of American oligarchs or Chinese cadres. I want European values of enlightenment and humanitarianism to be the default.

DeathArrow 3 hours ago | parent [-]

>I don't want my model to reflect the views of American oligarchs or Chinese cadres. I want European values of enlightenment and humanitarianism to be the default.

I don't want AI models to reflect any kind of values whatsoever. I have my own views - thank you - and I don't need other throwing their values in my face using AI.

I prefer AI models not being trained ideologically.

wccrawford 3 hours ago | parent | next [-]

Unfortunately, AI models are trained on humans, and so they're getting that whether we like it or not. I'd also prefer it to be neutral, but I don't think it's possible and the second-best option is to have models from multiple regions and balance things out.

vrganj 3 hours ago | parent | prev [-]

There is no unideological. There is no view from nowhere, what you perceive as unideological is precisely the reflection of your own ideology. The "unbiased" is just whatever matches your own bias.

To quote Zizek:

> I already am eating from the trashcan all the time. The name of this trashcan is ideology. The material force of ideology - makes me not see what I'm effectively eating. It's not only our reality which enslaves us. The tragedy of our predicament - when we are within ideology, is that - when we think that we escape it into our dreams - at that point we are within ideology.