| ▲ | origini an hour ago | ||||||||||||||||||||||||||||
I would counter argue with "But chinese models use distillation of Frontiermodels..." which is different than being a Frontierlab or not clear how different it is. But Microsoft coming from the side like this might be very different if it holds up. | |||||||||||||||||||||||||||||
| ▲ | adrian_b an hour ago | parent | next [-] | ||||||||||||||||||||||||||||
For now I have not seen any credible evidence for "But chinese models use distillation of Frontiermodels...". Of course, I would not be surprised if they also do this, mainly because after OpenAI and Anthropic have succeeded to gather most training information that was easily accessible in the past and now it has become much more difficult to do the same. For some information that was hoarded by OpenAI and Anthropic, public access may have become impossible otherwise than through their LLMs. Nonetheless, distillation cannot be the main source of improvement for the Chinese models, among other reasons because even assuming that they may have access to a great number of fake accounts, there still is not enough time for a lot of distillation, taking into consideration the timeline of the releases, and it is not believable that the attempts to do distillation would not be detected quickly by OpenAI and Anthropic, leading to the banning of the fake accounts. While OpenAI and Anthropic provide negligible information about their models, the Chinese AI companies are much more transparent about their methods and we can see that there is a lot of diversity among their models and a lot of innovations with each new version. | |||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||
| ▲ | eli 41 minutes ago | parent | prev [-] | ||||||||||||||||||||||||||||
Even if OpenAI/Anthropic remain "a few months" ahead of open weight models indefinitely that still is not great news for them. | |||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||