| ▲ | himata4113 an hour ago | |
It's not that their strategy is to train smaller models, it's the only choice they have. Training SOTA takes anywhere from 1.5b to 150b. We don't know the real cost of training for the chinese models, but mistral neither has the compute nor money to do that. | ||
| ▲ | maelito 2 minutes ago | parent [-] | |
Do you have a reference explaining these costs ? Part by part. | ||