One of the explicit goals of the NSF is to train the next generation of scientists. Part of that is making sure that you're creating a rich pipeline of people who are going to do innovative things. Broadening participation is much more about things like getting more (usually younger) people from all walks of life interested in joining your field. Which is basically an unmitigated good -- first, the obvious advantage that having more people who want to be in a field is good for it from the perspective of choosing the best folks. And second, the less obvious but perhaps more important thing that people with different perspectives often end up thinking about problems differently. It's not nearly as helpful to have 1000 people all focused on chasing the same problems with the same toolbox of solutions as it is to have 1000 people focused on different problems with different ideas of how to approach them.

I say this as a professor at a top computer science department. I have _never_ felt limited in my ability to collaborate with the best folks in my area. Ever. I do! And it's great! And I also believe strongly it's important to make sure we are growing those next generations of amazing people, because the thing that makes research awesome is working with them.