Using artificial neural networks directly for information storage and retrieval (i.e. not just leveraging them as tools accessing other types of storage) is currently infeasible, agreed.

On the other hand, biological neural networks are doing it all the time :) And there might well be an advantage to it (or a hybrid method), once we can make it more economical.

After all, the embedding vector space is shaped by the distribution of training data, and if you have out-of-distribution data coming in due to a new or changed environment, RAG using pre-trained models and their vector spaces will only go so far.

eh, bio neural networks aren't doing that all the time. Memmories are notorious for being "rebuilt" constantly.