| ▲ | pizza 2 days ago | |||||||
They do have a weak relationship, in that earlier index tokens were encountered earlier during the formation of the vocabulary, so they are similar in typicality | ||||||||
| ▲ | janalsncm a day ago | parent [-] | |||||||
No, if you check the diagram (page 2) these are literally indexes into the KV vectors, not positional indexes in the text. If it was the text I would agree with you. | ||||||||
| ||||||||