▲ | yosefk 10 days ago | |
Actually I forgive them those issues that stem from tokenization. I used to make fun at them for listing datum as a noun whose plural form ends with an i, but once I learned about how tokenization works, I no longer do it - it feels like mocking a person's intelligence because of a speech impediment or something... I am very kind to these things, I think | ||
▲ | astrange 8 days ago | parent [-] | |
Tokenization makes things harder, but it doesn't make them impossible. Just takes a bit more memorization. Other writing systems come with "tokenization" built in making it still a live issue. Think of answering: 1. How many n's are in 日本? 2. How many ん's are in 日本? (Answers are 2 and 1.) |