I think this partially buries the lede: "As a single hiring vendor comes to dominate screening for an industry, it may be more likely that candidates are shut out."

If we move to using just a small number of AI models to help do things like hiring, we will amplify biases and possibly completely lock out portions of the population. We need to be very careful when using AI systems to evaluate people in general -- not because they might be biased (which they might be), but because even a small bias, if used by virtually everyone, can be damning.

> We need to be very careful when using AI systems to evaluate people in general -- not because they might be biased (which they might be), but because even a small bias, if used by virtually everyone, can be damning.

I don't think this even requires any bias.

Assume there's some loose ordering of who is or isn't a good hire, and every employer has their own fuzzy view of it. If you get slightly better or worse as a potential hire (pick up an extra degree, let your latest certification lapse, whatever), it gets somewhat easier or harder to get hired.

Now assume that same ordering, but all employers share the same view of it. I'd expect the divide between employable and not employable to be much sharper.

It's strange to say it might be biased. Bias is absolutely impossible to avoid, especially with how today's "AI" works.

You might be able to avoid it with a panel of AI, similar to how we try to avoid it by using panels of humans, but even that turns out to be contentious and not surefire.

I have feeling with AI it'll be even worse, since folks / companies can pass the buck (similar to how health insurance companies are now using it to deny folks).

If you want to make meaningful change in this avenue you really can't use words like "bias" or "systemic" because anywhere from 49-51% of the population will immediately shut down upon hearing that. Someone can argue (and many do to varying levels of success) that systemic bias doesn't exist, which means this doesn't exist, which means there no problem.

However, "this AI model can decide that some subset of people, perhaps random, perhaps not, are simply not hirable for any job" makes sense to most people regardless of political bent.

Agreed. Humans are also biased, but our biases are different across a lot of socio-economic factors. So when we have different people in these positions, the biases become less bias-y.

But LLMs are statistical models. They are aggregating all biases into a general super bias. And they're all converging towards the same solutions.

It is also illegal