| ▲ | dahart 2 hours ago | |
Looking at the first several examples, shorter (one, two, three-) letter words are not stretched to the full ‘fixed’ width. It’s the whitespace around them that is fixed. The only example in the first three screenshots of splitting a word into tokes is ‘vibes’ which turns into ‘vib’ and ‘es’. I’m skeptical of the -ing and -tion words being single tokens as shown. | ||