Remix.run Logo
▲ robrenaud 5 hours ago

Use different tokenizers for different languages. Or at least, use different tokenizers for different scripts.