Remix.run Logo
fastball 6 days ago

Mistral models are not very competitive with other proprietary models. Their competition is mostly from OSS models, which 1. can actually be run anywhere and 2. frequently outperform Mistral models anyway (e.g. DeepSeek 3, Kimi K2, and Qwen3 all outperform Mistral in current LMArena rankings[1]).

Hell, you can host actual frontier models (e.g. Claude 4) on AWS Bedrock in the EU, so "in the EU" (from a hosting perspective) cannot be Mistral's USP. If the proposition is "support EU businesses", then ok, but that is a different thing.

[1] https://lmarena.ai/leaderboard/text

pimterry 6 days ago | parent | next [-]

> Hell, you can host actual frontier models (e.g. Claude 4) on AWS Bedrock in the EU, so "in the EU" (from a hosting perspective) cannot be Mistral's USP.

I've seen zero cases so far where "physically present & managed in the EU but still owned by a US company" is sufficient to mitigate the typical US hosting concerns.

The threat is that AWS could be forced to a) suddenly pull services or b) spy on data by the US administration. That the DC is located entirely in the EU does nothing to reduce that risk if it's still fully owned by Amazon.

The was already a major concern for the last couple of years given the successful legal challenges against the privacy shield as sufficient data protection to give personal data to US organizations, and is way more of a concern after issues like Karin Khan and the ICC being suddenly cut off by Microsoft - it's clear that US companies literally can & will suddenly block key business services on administration whims. There's plenty of organizations where that's unacceptable risk.

epolanski 6 days ago | parent | next [-]

> I've seen zero cases so far where "physically present & managed in the EU but still owned by a US company" is sufficient to mitigate the typical US hosting concerns.

I did. Some of my clients by design host everything on German servers of Azure and call it a day.

f_devd 6 days ago | parent | next [-]

To be fair Microsoft has put the most effort[0] of any US company I've seen in order to try and work around the issue. Not that I would choose it.

[0]: https://www.microsoft.com/en-us/trust-center/privacy/europea...

sunaookami 6 days ago | parent [-]

As long as the CLOUD act exists (which to their credit Microsoft fought) there is no privacy as long as there are US companies involved.

ta20240528 6 days ago | parent | prev | next [-]

Then your customers are morons, let me explain:

1. the USA has secret FISA courts - defendants cannot even say they whether they were summoned, let alone what case or judgements were

2. the CLOUD Act compels American companies to hand over data, regardless of where its hosted.

So your German companies would never even know if they have been compromised.

But ignorance can be bliss.

pyrale 6 days ago | parent | prev | next [-]

> Some of my clients by design host everything on German servers of Azure and call it a day.

Accepting the risk isn't the same as finding a way to mitigate it. Plenty of EU companies just happily use US cloud providers, that doesn't mean the risk doesn't exist.

croes 6 days ago | parent | prev [-]

Did you correct them or do you wait until they get a warning letter from some shady law firm for violating GDPR?

epolanski 6 days ago | parent [-]

This is beyond my paycheck and responsibility to be honest.

jansenmac 6 days ago | parent | prev [-]

Microsoft didn't cut off Karim Khan. https://www.politico.eu/article/microsoft-did-not-cut-servic...

pimterry 6 days ago | parent [-]

That's not what that article says - it says they didn't completely cut off service to the entire ICC. The headline is confusing, but the quotes are pretty clear:

> A Microsoft spokesperson said that it had been in contact with the court since February “throughout the process that resulted in the disconnection of its sanctioned official from Microsoft services."

cloudify 6 days ago | parent | prev | next [-]

Due to the Cloud Act, hosting "In the EU through a US company" and "In the EU through an EU company" are two very different things.

fastball 6 days ago | parent [-]

Sure, but that's why I mentioned all the open models which are better than Mistral and you can run where and however you want.

epolanski 6 days ago | parent | prev | next [-]

> Mistral models are not very competitive with other proprietary models.

As an enterprise user of various models, this is absolutely wrong and false.

What matters when using models as a service is:

- type of work involved

- speed

- cost

- law compliance

And, believe it or not your benchmarks IRL are worthless for most of the things you want to give to AI (unless we talking about coding idk).

I'll provide you few examples where Mistral is by far the best option for our companies from applications in production, even ignoring the last one.

- customer care assistance. One of my clients is in the business of home renovation, customers call the company to have details about how to install/mount specific things. For my use case: OCR + information retrieval from the scanned documents + reporting to our assistancs Mistral displayed by far the best performance (they have the best AI OCR we tested) and cost effectiveness and speed.

- creating user-tailored daily financial news. We need to summarize, rank and report what happened for user-held securities during the day. The only competitive alternative here to Mistral was Google's Gemini Flash, we need to do this for tens of thousands of users. Mistral Small was absolutely up to the task, with the Medium variant for ranking and bundling. We have tested the other options and literally nobody offered the same performance/cost/speed

fastball 6 days ago | parent [-]

FWIW in my own testing I found Gemini OCR better than Mistral.

BoredPositron 6 days ago | parent | prev | next [-]

At the risk of being contrarian, investment decisions are rarely driven by publicly available product offerings alone.

6 days ago | parent | prev | next [-]
[deleted]
mosselman 6 days ago | parent | prev [-]

I know they aren’t. So I am hoping this investment will change that.