Processing 500 retrieved chunks here (probably with an additional LLM) will also add a bit of latency (not shown in these graphs!).

So for user facing apps, that scenario is probably not feasible (more like filter 10 chunks). Which as the parent of this comment suggests is fine to add in extra context given the current size of context windows.