There's a large body of research on context pruning/rewriting (I know because I'm knee deep in benchmarks in release prep for my context compiler), definitely don't ad hoc this.

▲

spariev 4 days ago | parent [-]

Care to give some pointers on what to look at? Looks like I will be doing something similar soon so that would be much appreciated

	▲	CuriouslyC 4 days ago \| parent [-]
		Just ask chat gpt about state of the art in context pruning and other methods to optimize the context being provided to a LLM, it's a good research helper. The right mental model is that it's basically like RAG in reverse, instead of trying to select and rank from a data set, you're trying to select and rank from context given a budget.