The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

Vector embeddings predate LLMs. They have been used as far back as the early 2000s. They are a general machine learning technique, rather than LLM specific

Sure and that's a new technique for indexing and querying.

Where's the new design tension? Indexes always had to be monitored for freshness and queries have always needed cleaning or parsing.

right, it is the foundation of machine learning.

not really, vectorising text/books is old school ML by this point.

at least to me that seems the same as https://en.wikipedia.org/wiki/Word2vec for e.g.