▲ | yorwba 12 hours ago | |
> You don’t need all the status quo bias of null hypothesis testing. You don't have to make the status quo be the null hypothesis. If you make a change, you probably already think that your change is better or at least neutral, so make that the null. If you get a strong signal that your change is actually worse, rejecting the null, revert the change. Not "only keep changes that are clearly good" but "don't keep changes that are clearly bad." | ||
▲ | scott_w 11 hours ago | parent [-] | |
This is a reasonable approach, particularly when you’re looking at moving towards a bigger redesign that might not pay off right away. I’ve seen it called “non-inferiority test,” if you’re curious. |