> The second mistake they made is assume that companies would prioritize being lean and trimming the mediocre & bottom 5%. There are other considerations, combined productivity is more important than having individual superstars working on the shiniest features.

I'll add a perverse incentive too that I've talked about elsewhere – hiring is a goddamn mess right now.

If I trim the bottom 5% of my org (in my case, 2-3 engineers), I may not get a backfill for them. Or I'll have to drop their level from L5->L4 to make finance happy, or hire overseas or convert a FTE to a contractor.

I also have to be ready for the potential of RIFs happening, which means having an instantly identifiable bottom 5% puts me at the advantage of being ready when my boss says "give me your names".

So the time value of a staffed engineer is way higher right now than it might be in a few months. It'll never be zero, because proactively managing people out makes all of our managers happy. But for now, I definitely need my low performers.

Completely agree. Yes, great engineers tend to be compensated well, but only slightly more than the median performer. In other words, a 10x engineer isn't getting paid 10x more. It's probably more like 1.5x to 2x. If we somehow invent a magical way to track productivity numbers exactly, I suspect we'd see something closer to Price's Law [1] (Pareto distribution) which is essentially what this post is about, where something like 20% of workers contribute to 80% of the results. However, that doesn't means the other 80% is expendable which relates to your second point.

Paying what employees "earn" for the company is incompatible with our economic system where companies want to be profitable. Paying employees what they "deserve" based on contribution is probably also undesirable. I think you'd get the same income inequality dynamics but within companies. There is an averaging effect when you work at large corporations. That's either a good thing or a bad thing depending on the person. Individual contributions are averaged out, but so are responsibilities. I think Paul Graham articulated this wonderfully in his essay on what a job is and why some prefer to work for startups [2].

[1] https://www.kienbaum.com/blog/prices-law-and-the-trouble-of-...

[2] https://paulgraham.com/wealth.html.

It's also just embarrassing that this is supposed to be a data science blog about employee performance and the only non-simulated data directly presented or discussed is the US wage distribution, where the author has just cavalierly marked the x-axis as "Performance". There's all this spew, and the author makes claims about what good data scientists do ... and there's no data in this discussion that's directly relevant to their rambling claims.

I have a crazy idea.

If a corporation lays off any people in a particular job category/title, that corporation should not be allocated ANY H1B visas for that job category/title for the next year.

If a corporation institutes any policy that requires decimation (or any other statistic-based termination program) of employees with a particular job category or title, or if IN EFFECT they perform this (because they will just hide it otherwise), then they will not be allocated any H1B visas for that job category or title, for the next year following any such act.

In essence, the point here is that if a corporation decides it can live without X% of their workforce, then they don't get to go bring in foreign workers. The H1B program is to help find workers for positions that can't be filled; if you're laying off or mass firing people then obviously you CAN find people to fill those jobs.

There is arguably no way to judge "fairness". Employee A gets assigned to work on LLMs. Employee B gets assigned to work on Android 9/Mac OS 12 security patches. Another example includes unforeseen difficulties. I have a friend that signed up to implement a feature. That feature would have taken 1-2 weeks in any standalone app, but, he happened to be on web browser team and the number of edge cases that came up and the amount of back and forth between standards committees meant it ended up taking 2 years. He was judged poorly even though all of it was out of his control because everyone though it should have taking 1-2 weeks.

I feel like I'd prefer some balance. There are superstars. We know them. We can easily point them out in peer reviews. But, we're also a team, there's lots to do, not everyone gets to work on the high profile easy to identify "impact" parts.

>. How much revenue do you think a janitor or café staffer generates? Close to zero. The same goes for engineering. Someone has to do the unglamorous staff, or you end up with a dysfunctional company, with amazing talent (on paper).

There's two ways to make a profit. Gain more revenue, and not lose more revenue. Those kinds of staff are the latter, in addition to other aspects like HR (preventing lawsuits/settlements which are expensive).

But yes, there's so many hidden factors on measuring "productivity". That's why stack ranking is a bit stupid in the long run. Some people aren't just producing value but bringing out productivity in others. But that's an opportunity cost for a stacked system. Such individuals should be considered for management, not kicked out.

>The two tracked closely, rising in tandem until the 1970s, where they got decoupled. With income becoming much flatter, and productivity continuing to rise. That's how the world has been for the past 50 years on the macro and the micro

Yup, very well known that we really should be close to that ideal John Maynard Keynes predicted all the way in 1930 of 15 hour workweeks by 2030. Instead, I believe the average work week in the US is 50 hours and it's still a very controversial battle to get to a 4 day work week.

Employers want to pay the minimum, clearly, but until a person’s salary exceeds the value they bring to a firm, there will be other firms willing to pay more and attract that talent. So provides some upward pressure on wages, which the author addresses:

> Economists will teach you something called the Marginal Productivity Theory of Wages, the idea being that the amount of money that a company is willing to spend on an employee is essentially the value that the company expects to get out of their work. This strikes me as mostly true, most of the time, and likely to be the case in the corporate world that we’re considering here.

The thing is, even if performance was Gaussian, even if we run with the following 2 statements:

- IQ is gaussian

- IQ correllates well with performance

hiring practices would probably produce an employee population that went through some right-curve cutoff test, meaning most people would be much closer to the hiring threshold, with a few positive outliers.

For a given arbitrarily chosen values, you could massage the distribution and make it look Pareto, but I'd be hard pressed to come up with a reason why it makes rational sense.

> Employers strive to pay employees the minimum they can bear, on employer's terms.

This is one of the things I try to drive home when I mentor young people. Employment is a market and it responds to the forces of supply and demand. Never think that your relationship with a company is anything other than a business transaction.

It's a hard lesson for young people to accept these days, but everything becomes much more clear once you stop fighting the idea.

I believe the graph you're referring to is here: https://wtfhappenedin1971.com/

> mistakes: the first is they assume employees are (or should be) paid according to how much they "individually" earned the company

From the article:

  Economists will teach you something called the Marginal Productivity Theory of Wages, the idea being that the amount of money that a company is willing to spend on an employee is essentially the value that the company expects to get out of their work. This strikes me as mostly true, most of the time

  The marginal productivity theory of wages states that under perfect competition, workers of the same skill and efficiency will earn a wage equal to the value of their marginal product. The marginal product is the additional output from employing one more worker while keeping other factors constant. However, the theory has limitations as it assumes perfect competition, homogeneous labor, and other unrealistic conditions. In reality, competition is imperfect, labor is not perfectly mobile, and other factors like capital and management efficiency affect productivity.

When I first heard of the Work Number, I thought there's no way they stay in business given the Real Page suit.

My takeaway ( and an indication of who actually needs a performance review [ e.g. the manager ])

“ It’s my opinion that the biggest factor in an employee's performance – perhaps bigger than the employee’s abilities and level of effort – is whether their manager set them up for success “

> Employers strive to pay employees the minimum they can bear, on employer's terms

I don't think it's worth thinking like this. An employee's salary is floored by their value to any company, and ceilinged by their value to the company currently employing them.

If you look at those graphs you’ll see a more telling tale. US productivity has skyrocketed but Europeans are not that productive in comparison and are stagnant. Might help with understanding what’s going on.

> How much revenue do you think a janitor or café staffer generates? Close to zero.

This completely depends on how you do your internal revenue accounting.

> the first is they assume employees are (or should be) paid according to how much they earned the company

From the perspective of a employee and/or human, that does seem like the most fair way of distributing what the company earns, sans the money that gets reinvested straight back into the business itself. But I'd guess that'd be more of a co-operative, and less like the typical for-profit company most companies are today.

About your last "edit" paragraph: You can read more about it here: https://www.epi.org/productivity-pay-gap/

TL;DR: Productivity–Pay Tracker: Change 1979q4–2024q1; Productivity +80.9%; Hourly pay: +29.4%; Productivity has grown 2.7x as much as pay

Also, if people are unfamiliar with economics term "productivity", it roughly corresponds to profit per employee, or better, return on equity (ROE).

That “infamous graph” is mostly pop nonsense. Simply google productivity graph versus income debunk economists.

The problem is bad Econ like that latch onto certain untrained brains like quack cures to an antivaxxer, because they reinforce unfounded beliefs (beliefs formed by eating too much of stuff like this).

The actual items under consideration are far less spectacular or supporting the dearth of conspiracies, so the truth doesn’t spread as fast. It’s not so shiny or conspiracy reinforcing.

This is a subtle aspect of intelligence measurement that not many people think about.

To go from an IQ of 100 to 130 might require an increase in brainpower of x, and from 130 to 170 might require 3x for example, and from 170-171 might be 9x compared to 100.

We have to have a relative scale and contrive a Gaussian from the scores because we don’t have an absolute measure of intelligence.

It would be a monumental achievement if computer science ever advances to the point where we have a mathematical way of determining the minimum absolute intelligence required to solve a given problem.

IQ scores have proven highly correlated to educational achievement, occupational attainment, career advancement, lifetime earnings, brain volume, cortical thickness, health, longevity, and more.

To the point of being accurate predictors of these things even when controlling for things like socioeconomic background.

It's used because it works as a measuring tool, how the tests are constructed is largely irrelevant to the question of if the outcome of the test is an accurate predictor of things we care about.

If you think you have a better measuring tool you should propose it and win several awards and accolades. No one has found one yet in spite of many smart people trying for decades.

Correct. IQ isn't an effective measurement of intelligence as is typically stated. It is (at best) a measurement of learning disabilities.

I didn't know that about how IQ tests are formed. Would that mean that there could be some sliver of the population that could score in the top %'s on the 1000 question test but due to the selection of questions, scored average on the final IQ exam? If so, that'll be my excuse next time I have to take an IQ exam. I just got the wrong distribution.

Sum of N independent similarly distributed variables (questions), will tend to be normally distributed, that the central limit theorem, no need to manufacture anything.

> and then pick out 100 questions that form a Gaussian distribution. This is how IQ tests are created.

You missed an extremely important final step. People's scores on those 100 questions still aren't going to form a Gaussion distribution. You have to rank-order everyone's scores, then you assign the final IQ scores based on each person's ranking, not their raw score.

It's worse, because every test is obviously bounded, and it's absurd to not expect some noise to be there.

Join those two, and the test only becomes reasonable near the middle. But the middle is exactly where the pick of questions makes the most difference.

All said, this means that IQ is kinda useful for sociological studies with large samples. But if you use it you are adding error, it's not reasonable to expect that error not to correlate with whatever you are looking at (since nobody understands it well), and it's not reasonable to expect the results to be stable. And it's really useless to make decisions based on small sample sizes.

That’s not how IQ tests are made as can be found by reading how they’re actually made via Google scholar. And it would be spectacularly hard to do what you describe.

How they’re actually made is a batch of questions thought to take some form of reasoning are curated, then ALL of those questions are used in the test. It is an empirical fact the percentages of decent sized groups of people will score a bell curve, in exactly the same way humans do on hard calc exams, on hard writing items, on chess problems, and across a bewildering amount of mental tasks, none of which are preselected and fidgeted with to fake a Gaussian.

A simple example: see how many simple arithmetic problems people can do in fixed time. What do you find? Gaussian. No need to fiddle with removing pesky problem. Do reading. Do repeat this sequence for length. Just about any single class of questions has the same bell curve output in human mental ability. The curve may bend based on some inherent difficulty, say addition versus calculus, but there will be a bell curve.

Now take plenty of types of questions to address various wobbles in people’s knowledge, upbringing, culture, etc, giving a host of bell curves per category (and those also correlated by individual). Then the sum of gaussians is gausdian. All IQ tests do is shift the mean score to be called 100 (normalized) and the std dev to match a preset amount of people so such tests can be compared over time.

And the empirical evidence is these curves do strongly correlate over time, so scaling a test to align with this underlying g factor is well founded.

This latter fact, that score on one form of intelligence seems to transfer well to others, forms the basis of modern intelligence research on the g factor. IQ tests correlate well with this g factor. And across all sorts of things the results are bell curves.

For anyone wanting to hear all this and a ton more, Lex Fridman has an excellent interview with a state of the art intelligence researcher at https://www.youtube.com/watch?v=hppbxV9C63g. The researcher goes into great depth on what researchers do know, how they know it, what they don’t know, and what has been proven wrong. This is all there.

People may find that manufactured or "oh IQ is just made up and there is no measure of intelligence". But I find beauty in the way that IQ tests create and reconfigure a distribution across a multi-dimensional vector or dimensional space. It figures out what we need in the general case, and allows us to use and reason with it, without ever having to do the grunt work or arguably impossible task of finding out an actual measure of intelligence or some way to untangle the way a brain works.

Yes, this has always bothered me. IQ doesn't easily correspond to any measurable real-world quality.

For example, if we would postulate that height is gaussian, we could measure people's heights and just ordering them we could create a gaussian distribution. Then we could verify the hypothesis of height being gaussian by mapping the probability distribution function's parameter to a linear value (cm) and find that these approaches line up experimentally.

We could do the same thing with any comparable quantity and make an order of them and try to map them to a gaussian distribution, but we would have no knowledge if what we were making actually corresponded to a linear quantity.

This is a serious issue, as basically making any claim like 'group A scores 5 points higher than group B' is automatically, mathematically invalid.

I think your comment about an easy test having everyone in the “knows a lot” category hints that the reverse (a hard test) would be Pareto distributed.

To me it says that our system is built on a reasonable but untested assumption (performance is a gaussian) and by replacing it with an equally reasonable assumption (performance is a pareto), suddenly our system looks stupid. It isn't really offering a solution but a new perspective

I thought that Bonus Content #1 and the references down the article were reasonably convincing. It would be great if large companies disclosed such details but it is unlikely.

> I would feel better if this was derived from empirical data rather than just rhetoric.

This exact statement applies to the practice of Gaussian performance ranking. It is pure corporate politics, it isn't founded in sound statistics.

The present author at least provides multiple sources of statistical evidence for their beliefs, if you read the footnotes.

The problem is that intellectual productivity is generally not possible to measure directly, so you instead end up with indirect measurements that assume a Gaussian distribution.

IQ is famously Gaussian distributed... mainly because it's defined that way, not because human "intelligence" (good luck defining that) is Gaussian.

If you look at board game Elo ratings (poor test for intelligence but we'll ignore that), they do not follow a Gaussian distribution, even though Elo assumes a Gaussian distribution for game outcomes (but not the population). So that's good evidence that aptitude/skill in intellectual subjects isn't Gaussian (but it's also not Pareto iirc).

Agree with you - although, rhetorically speaking, I have come across many instances which the author refers to "of low performers are 3x as common as high performers." This is unfortunate as I always think do your best, and as Tyler Cowen states - Average is Over. So agree it would have been way better to use empirical data to back up this claim especially.

> I think that any of these simplified distributions are likely not generalizable across companies and industries

It’s going to be multivariate statistics with dependent variables. The quality of non developers at company affects the quality of developers they can retain, and the quality of the developers you have affects the quality of developers you can recruit and improve. Almost all the people I’d want to work with again left my last employer before I did.

You can take on more and more work yourself but it causes everyone around you to disengage. At some point you have to realize it’s more fruitful, emotionally and mathematically, to make coworkers produce one more unit of forward progress a month than to do it to yourself. Because it’s 2% for the team one way and 5-10% the other.

> There is probably a ton of data already in different industries with regards to productivity.

Uh. Not really. Our industry is notoriously bad at measuring productivity.

And the bigger problem is that when we try to measure it - "performance review" - we like grading on a gaussian curve. We'll never know if that's correct because we put our thumb on the scale.

An even bigger problem is that productivity is strongly influenced by completely non-technical factors. How enthusiastic are folks about what they are doing[1], how much variety do their tasks have [2], what are their peers like, etc. (Of course, that whole field of study has issues rooted in the inability to measure precisely as well)

Ultimately, it's a squishy judgment applied by humans.

[1] https://www.semanticscholar.org/paper/What-Predicts-Software...

[2] https://research.google/pubs/what-predicts-software-develope...

It's really difficult for me to believe that they really got 10% top performers. For one, knowing the cut-throat nature of employment there, I would expect only a minority of developers would be willing to try working there, despite the awesome rewards.

Another reason I really don't trust that to be true is that I've never seen a good way to measure who is a top performer and who is not. I don't think there's one, people are good in different things, even within the same job... for one assignment, Joe may be the best, but for another, Mary is the winner (but again, to measure this reliably and objectively is nearly impossible IMHO for anything related to knowledge work - and I've read lots of research in this area!).

Finally, just as a cheap shot at Netflix, sorry I can't resist as a customer: they absolutely suck at the most basic stuff in their business, which is to produce good content in the first place, and very importantly, NOT FREAKING CANCEL the best content! I won't even mention how horrible their latest big live stream was... oh well, I just did :D.

In summary, Netflix told all their employees that they are so amazing at their job, they are the top 10% of the whole world, they are like NFL athletes. If they don't perform to top tier levels, they'll be shown the door.

Here's a thought experiment: pretend that Netflix is lying and that their employees are not actually made up of the top 10% of talent industrywide. Let's for this thought experiment assume the realit is that they have slightly above average talent because Netflix pays slightly above industry average.

But now they've convinced those employees that they're not just slightly above average, they are like elite NFL players. And that means they have to work like elite NFL players. Netflix convinces their employees to work XX% harder with longer hours than the rest of the industry because they think they are elite.

"Only amazing pro athlete geniuses can work here" is way more motivating than "You have to work yourself to death with extra hours to make quota or you're fired!" because it's a manipulation of the ego.

I think this thought experiment is closer to reality than Netflix or their kool-aid-drunk employees will admit, and that Netflix's "pro athlete" culture is worker-harming psychological manipulation.

How are 'top performers' and 'low performers' being defined in this context?

In my experience, these labels in corporate environments often correlate more with social dynamics and political acumen than actual work output. People who are less socially connected or don't engage in office politics may find themselves labeled as 'low performers' regardless of their actual contributions, while those who excel at workplace networking might be deemed 'top performers'.

The interview process of these kind of companies also often falls into a problematic pattern where interviewers pose esoteric questions they've recently researched or that happen to align with their narrow specialization from years in the same role. This turns technical interviews into more of a game of matching specific knowledge rather than evaluating problem-solving abilities, broader engineering competence or any notion of 'performance'.

Let's be honest: how many people can truly separate personal feelings from performance evaluation? Even with structured review processes in place, would most evaluators give high marks to someone they personally dislike, even if that person consistently delivers excellent work?

A bit offtopic, but I've been curious about this.

Could you please describe how the unlimited vacation policy worked? How did people feel about it and whether they were anxious regarding using it (afraid that it will reflect on them badly when they take "too much" time off)?

Logically no-one else will try this - because if every company competed for the top talent, wages would rise to consume all profit.

I think this is probably how labour and capital should compete - I expect we need to equalise tax treatment so that becomes more possible

> Performance and compensation were completely separate, which was also nice.

Huh? How is that nice? Does performance and compensation not correlate in your ideal world, or am I misunderstanding it?

How can 360 peer performance reviews ever work? The incentives are against a fair evaluation: the reviewers have the incentive to overly criticize others so that they can stand out more.

I'm not saying that everyone on a 360 review process does that. But the incentive is there and it's working against fair reviews.

For analyses like this it just doesn't matter. Pick a metric and measure it over your workforce. Across the universe of salient metrics of interest you won't see a gaussian across your workforce.

In a previous job I modelled this and concluded that due to measurement error and year-over-yead enrichment, Welchian rank-and-yank results in firing people at random.

Oh, the answer to that is apparent enough, but frustratingly circular:

Performance is "visibly doing the things that the company rewards during the performance review process".

Theoretically, each role at a company should have a set of articulated accomplishments that are expected. (This is sadly often not the case.)

But you're right that the subjective nature of "performance", and the lack of a clear numerical scale, are a difficulty of the entire process!

You could replace "employee performance" with "value to the company" and the same argument would hold. Performance is difficult to measure, but we get a good estimate of value to the company any time someone receives a competing offer and drags their manager to the negotiating table.

The amount of money the manager is willing to match is the perceived value to the company. This is how the company actually behaves (we know for sure whether they match the offer or not) and that behavior implies a value to the company, regardless of what anyone says in performance review season.

The article does briefly caution about measuring difficulties. But given that the main conclusion is an argument against stack-ranking-and-firing, the question of "what is performance" passes forward to whatever metric the stack-ranking manager was going to use when they were planning to fire the "bottom" 10% of their payroll.

I'm not sure this is the argument the author is making, but you could claim that the rest of the argument is true for any (or most) reasonable measure of employee performance that a company actually cares about.

On a meta note, you're right to note that unclear terms undermine our collective reasoning, despite a proper chain of propositions.

I've found Term Logic[1] to be useful for figuring out why certain discussions confuse me. I've also used to avoid unnecessary arguments by seeing if the participants are starting with clear concepts (signaled by terms).

[1] https://en.wikipedia.org/wiki/Term_logic#Basics also this explainer https://adoroergosum.blogspot.com/2015/05/the-three-acts-of-...

It also assumes that "productivity" is something that is meaningful at all at the level of individuals, not teams or larger. IMHO, it is not.

Yes, is performance Pareto, or perception of performance Pareto?