>Text tokens are high-dimensional vectors,

You are conflating tokens with embeddings.

Tokens fit in a single word, modern gpt uses a vocabulary with 200k possible values, which would fit into 18 bits.

Have a good one