| ▲ | sbinnee 6 hours ago | |
It is a good approach to promote it as a German model. But I wonder if it really thinks like the German think. I speculate they just translated texts in other languages, likely English and Chinese, into German. | ||
| ▲ | ivo-42 2 hours ago | parent | next [-] | |
Hey, I worked on pre-training data Kolibri. We spent considerable time and effort to go beyond just translating. For example by building a pipeline that processes Common Crawl dumps specifically for German. You might be interested in a related blog post: https://aleph-alpha.com/en/blog/sauerkraut-not-burgers-why-g... We also talk more about German pre-training data in the tech report: https://aleph-alpha.com/downloads/tech-report.pdf section 2.3.2.2. | ||
| ▲ | te0006 2 hours ago | parent | prev [-] | |
Read TFA. They were very aware of the dangers of such an approach. So they avoided it to the extent possible. | ||