There are quite a lot of approaches you can take to reduce the cost of this, e.g. sharding by search term, so the number of shards hit for any specific search term is a subset of the total set.

You can also certainly broadly cache the "top" of the hit lists for very common searches, so you don't need to fan out unless you're doing less common searches or going beyond the first "page" of results.