Im not full read up on RAG pipelines, but has anyone ever tried to make the database a neural net itself? I.e get rid of any sort of traditional databases, and then you basically just have some sort of autoencoder?

There's been quite a bit of research into this over the past 3-4 years under the name "generative retrieval." The general approach is to use a transformer and treat the weights as the index. You input the query, and then used constrained decoding to generate the document ID.

In some sense there's probably a database compression scheme that does something similar. Usually people care too much about fidelity