Remix.run Logo
daft_pink a day ago

Anyone who’s done hiring wouldn’t be shocked by this:

We find applicants are more likely to be rejected from every position they apply to than would be predicted by the baseline of each position making statistically independent decisions.

Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer. Like online dating, most employers are looking for some baseline indicators that you are going to be successful and stable.

verteu a day ago | parent | next [-]

> Obviously a rejected resume is more likely to be rejected by every other employer and an accepted resume is more likely to be accepted by every other employer.

But that wasn't the case for non-algorithmic screening. From the paper:

"By contrast, we find that when first round screening is not mediated by a single screening procedure, systemic rejections are close to the baseline. To support the empirical validity of our baseline, we study homogeneous outcomes in the largest study of first-round screening at U.S. employers to date. Kline et al. [38] generated 83000 synthetic resumes and submitted these resumes to vacant positions at 108 US companies between October 2019 and April 2021, a similar time period to our data. The companies, which are a subset of the Fortune 500,15 collectively employ 15 million workers. We analyze the homogeneity observed in the resulting callback outcomes in their data. We find that the baseline is an effective estimator of the systemic rejection rate for this dataset. As shown in Figure 3, the observed systemic rejection rate is accurately predicted by the baseline and a chi-squared goodness-of-fit test cannot reject equality of the two distributions (2 = 20.05, = 0.69). In other words, while the largest previous study observes systemic rejection rates consistent with employers making statistically independent decisions, the algorithmic hiring data shows significantly correlated outcomes that lead to higher-than-baseline systemic rejection rates."

tbrownaw a day ago | parent [-]

The paper with the non-algorithmic screening used synthetic resumes. I rather suspect that they didn't generate a realistic distribution of qualifications levels.

verteu a day ago | parent [-]

Yeah, good point.

drillsteps5 5 hours ago | parent | prev | next [-]

This does not sound right to me. It would be correct if all companies and their hiring managers had the same requirements/looking for candidates with the same qualifications - but they're not.

Obviously as a hiring manager you're looking for a hard working individual with a number of successfully completed projects and glowing referrals from multiple places of employment, but you're also looking for a person with expertise in particular technologies/industries/whatever other areas of expertise. To a large extent requirements for each role are unique, however many do have some overlap.

So being rejected from one position might simply mean there's a misalignment between what the company is looking for and what the individual has. Which might not be the case with other companies.

So if we're seeing increasing number of candidates being consistently rejected at multiple places the question "why" is a valid one.

zeroonetwothree a day ago | parent | prev | next [-]

Yes I don’t understand why this is surprising or problematic at all?

Actually the fact that they found this result didn’t hold in a different dataset is especially weird.

pc86 a day ago | parent | prev [-]

> a rejected resume is more likely to be rejected by every other employer

This makes sense to me, albeit intuitively and in a way I can't articulate.

> an accepted resume is more likely to be accepted by every other employer

but this doesn't necessarily follow from the prior for me. Plenty of people get really good jobs and are really successful in them only after dozens or hundreds of rejections with a nearly-identical resume.

daft_pink a day ago | parent | next [-]

If you look at the chart, the systemic rejection rate is only like 5-10%. It’s not a huge impact and it’s just about getting an interview not getting the job, so they could still get rejected.

I just think certain resumes will get an interview almost every time in some industries and certain resumes will likely never get an interview almost every time, but the majority of resumes are like you say have different aspects that appeal to one empoyer over another.

heylook a day ago | parent | prev [-]

The intuition is that they are not truly independent statistical events. Each trial reveals more information about the underlying "quality" of the resume (for passing this trial, not necessarily real world "quality" of the candidate). We are not rolling dice where each toss is fundamentally unrelated to prior tosses.