replace they key-query-value mechanic by just dropping it while making the entire context the latent space.

What do you mean by this? Like concatenating all token embeddings into one large vector?