The problem with A/B testing, and anyone who has ever done it at scale can tell you, beyond the banal basic stuff that people are already aware of like making things accessible and discoverable, once you get to a certain point, people just have no opinion. The default opinion is no opinion.

It's why every mass consumer product devolves into a feed or a list of content delivered by an algorithm. Once you reach a certain point, you come full circle and even that doesn't matter anymore: users will happily consume whatever you give them, within reason.

A/B testing platforms are mostly used by an odd collection of marketers and "data driven" people who love to run experiments and drag out every little change in the name of optimization. In the end, it all completely doesn't matter and doesn't tell you anything more than just talking to an average user will.

But, boy, are they sure a great way to look busy and dress up an underperforming product!

▲

beeon 5 hours ago | parent [-]

Maybe the industry you work in is relevant here. In e-commerce, A/B testing the position and color of "add to cart" button can yield legitimate revenue multipliers. People's opinions are irrelevant in that kind of A/B test, all that matters is the likelihood they continue down the funnel.

	▲	iamleppert an hour ago \| parent [-]
		This is always the straw man A/B testing people reach for. When in reality, this only works on the worst possible designs or bizarre layouts, that no one really uses. It's a myth that you can somehow ring out extra few % by making tiny changes to fonts and colors -- in every case, they simply stop the experiment when the results tip in their favor. The ONLY time I've ever seen it used successfully was to make changes to the layout of ads, making them look like more organic content or likely to be accidentally clicked. You can see this at work if you've ever clicked on an ad by mistake, or maybe you were trying to close the ad and noticed someone has "optimized" the close button placement using one of these A/B tools. There it is, that's the market for these tools. That represents the vast majority of the these companies use-cases, revenue and usage. Anyone else who implies an innocent intent is either ignorant or inexperienced.