| ▲ | rstuart4133 4 hours ago | |
> we have no access to the source materials and cannot reproduce the model independently. Given the USA companies have been loudly claiming the Chinese models are distillations of their models, also claiming "no access to source materials" seems dubious. As it was dubious anyway with because the Chinese publish lots of papers on how their models are designed, I'm left feeling I'm looking at the south end of a north bound bull. | ||
| ▲ | usef- 3 hours ago | parent [-] | |
Distillation of this kind isn't the only data that is fed into models, it's still a small minority. It might be weighed higher in training to learn the thinking patterns/etc but you need a lot more tokens. | ||