| ▲ | semiquaver 5 hours ago | |
Are there any advantages of the new tokenizer? Does it have a larger or smaller vocabulary or just differently weighted? | ||
| ▲ | ianberdin 5 hours ago | parent [-] | |
Well, in my view, it's just the most ordinary manipulation to avoid creating unrest. There is most likely no improvement inside. Of course, these are my guesses, but did anyone feel the difference in the transition from Opus 4.5 to 4.6? In my opinion, no. And it's unlikely to be a matter of the tokenizer. | ||