This sounds like when crystal people talk quantum physics.

I agree with the GP. The idea that there's not a better intermediate representation between tokens and embedding vectors seems absurd. But how to arrive at such a representation and implement it effectively is a few zeroes above my pay grade.

	▲	refulgentis 12 days ago \| parent [-]
		I find your agreement seductive because it side steps the unfounded assertions and simply asserts there must be something different and we don’t know it, which is easy for me to agree with too. Or maybe hard to disagree with.