The problem as I see it is that hiring the most qualified person for the job often requires DEI. That's because one of the primary goals of DEI programs is to attempt to ensure that people in different groups have the opportunity to demonstrate that they are qualified. That could take the form of trying to account for differences in how groups communicate their qualifications (e.g. certain neurodivergent people are likely to struggle processes that put excessive weight on ability to make small talk), trying to account for differences in access to opportunities to demonstrate qualifications (e.g. by sending recruiters to historically black colleges), trying to account for alternative was that people might be qualified (e.g. by trying to recognize how someone with technical experience form the military might be qualified even without a degree), or trying to avoid recruitment practices that are likely to favor people in the same group as the interviewer (e.g. being careful of basing hiring on "culture fit").

I'm not sure this is terribly relevant given where the conversation has gone, but in your example, college admissions, race was essentially used as a tie breaker between equally qualified candidates.

I suspect that's how it ended up being used in a lot of places (aside from deliberate outreaches to encourage applications, etc).

Beyond that though, I'm not sure not getting into harvard is exactly a "grave injustice". You don't have a right or entitlement to go to harvard regardless of what your academic score is. And I don't think there's a reasonable argument that there should be such a right.