| ▲ | mr90210 6 days ago | |
> survey of 791 developers We have got to stop. In a universe of well over 25 million programmers a sample of 791 is not significant enough to justify such headlines. We’ve got to do better than this, whatever this is. | ||
| ▲ | recursive 6 days ago | parent | next [-] | |
Validity of the sample size is not determined by its fraction of the whole population. I don't know the formulas and I'm not a statistician. Maybe someone can drop some citations. | ||
| ▲ | spmurrayzzz 6 days ago | parent | prev | next [-] | |
I generally agree with this just from a perspective of personal sentiment, it does feel wrong. But statistically speaking, at a 95% confidence level you'd be within a +/- 3.5% margin of error given the 791 sample size, irrespective of whether the population is 30k or 30M. | ||
| ▲ | oasisaimlessly 6 days ago | parent | prev [-] | |
You should read more about statistical significance. Under some reasonable assumptions, you can confidently certain deduce things with small sample sizes. From another perspective: we've deduced a lot of things about how atoms work without any given experiment inspecting more than an insignificant fraction of all atoms. TL;DR: The population size (25e6 total devs, 1e80 atoms in observable universe) is almost entirely irrelevant to hypothesis testing. | ||