Remix.run Logo
minimaxir 3 hours ago

A/B tests only work if the subjects don't realize they are in a A/B test.

abtinf 2 hours ago | parent | next [-]

Perhaps vibe coding the A/B testing engine isn't the best idea.

inetknght 2 hours ago | parent | prev [-]

Solution: don't A/B test your users.

A/B testing people without their informed consent is immoral, unethical, and should be illegal.

skeledrew 2 hours ago | parent | next [-]

To play devil's advocate, without A/B testing a lot of decisions would be made with insufficient relevant data, and lead to subpar results that affect the many negatively form the road.

wat10000 2 hours ago | parent [-]

A lot of decisions made with A/B testing are also made with insufficient relevant data, but it's less obvious since it's easy to think the A/B results cover everything.

shimman 2 hours ago | parent | prev | next [-]

Agreed and I can't wait until they regulate this stuff out of existence. It's absolutely hostile software technique that is deeply anti-human.

vehemenz 2 hours ago | parent | prev [-]

Depends entirely on the stakes and whether personal data is involved

inetknght 2 hours ago | parent [-]

> Depends entirely on the stakes and whether personal data is involved

Sure. Let me just A/B test whether or not you'll respond positively or negatively to having your news delivered via push notification or delayed by 10 minutes.

I'm sure you would appreciate being tested on without your consent, just so that I can make an extra quick buck at your expense. Nothing amoral or unethical about it.

pitched 2 hours ago | parent [-]

What do you think about slow rollouts for new features? Like, we think this new push notification system will be loved but let’s ship to only 1% of users in case there’s a horrible unforeseen consequence like occasional 10min delays? Dashboard goes upside down -> revert then work through logs to figure out what the hell went wrong.