Remix.run Logo
fastball 2 hours ago

Should've called it Safestral.

Also I do like Mistral's seemingly newer strategy of focusing on smaller, more fine-tuned models for various use-cases, presumably the result of their large MoE models not competing effectively with the frontier models.

himata4113 an hour ago | parent [-]

It's not that their strategy is to train smaller models, it's the only choice they have. Training SOTA takes anywhere from 1.5b to 150b. We don't know the real cost of training for the chinese models, but mistral neither has the compute nor money to do that.

maelito 2 minutes ago | parent [-]

Do you have a reference explaining these costs ? Part by part.