Dear Software Makers

OkayPhysicist an hour ago | parent | next [-]

I strongly believe A/B testing users without their knowledge and enthusiastic consent is unethical. If you don't have enough confidence in your changes to make them carte blanche, then don't make them at all. Users don't want their shit changing all the time.

	▲	sfRattan 7 minutes ago \| parent [-]
		> A/B testing users without their knowledge and enthusiastic consent is unethical. Yep, and A/B testing as experienced by uninformed, unaware end-users is a dark pattern. It undermines a perception of (and trust in) continuity which is necessary to make effective use of a tool. The best way I can describe it to the skeptical is: imagine the dials on your car's dashboard rearrange themselves occasionally overnight, and on some commutes to work you suddenly can't work the radio or the AC while moving at ≥35mph. Of course, since the widespread use of touchscreens, that example became very literal. So the car manufacturer has figured out the "optimal" arrangement of dials and buttons on their dashboard for their preferred levels of user engagement. Great. How many of those users now associate their car's brand with inconsistency? "I can't trust the damn buttons to be in the same place the next time I drive."

▲

akst 3 minutes ago | parent | prev | next [-]

In most cases the A and the B in an A/B test really should otherwise mostly identical, but one thing is different.

If almost everything is different, it’s hard to learn for next time what exactly what led to a change in which ever dependent variable your observating.

▲

smarf an hour ago | parent | prev | next [-]

> The skill of being a “creator” is not about maximizing views, retention, etc. It’s about finding creative ways to share stories, teach concepts, explore ideas, etc. Views, retention, etc. are all downstream of that.

this seems wildly naive

	▲	rglover 21 minutes ago \| parent [-]
		That's exactly what the focus should be. It may be naive in a modern context where everyone is addicted to "number go up," but it's the only way to get an authentic, not manufactured culture.

▲

smugengineer69 39 minutes ago | parent | prev [-]

I've seen this argument a few times and honestly I don't get it - how do people propose we otherwise measure relative success of particular interventions -- Vibes? I've found that people who complain about how everything needs to be measured and how measurements are imperfect are often just bad at devising measurable quantities as proxies of real-world benefit. Sure it can take a bit of time to devise such measurements, and they're not inherently valuable, but if I can prove that a given variant performs better, why not pursue the preferable alternative?

It seems easy enough to say "keep the youtube we love" but how do you think this variant came about in the first place? I can assure you there have been numerous A/B tests that have led to the current feature set. And even if its a local rather than global maximum, at least there are measurable qualities by which it is preferable. Also - do you believe that everyone who says this is harkening back to the same historical reality? This is quickly approaching "make youtube great again" territory - when exactly was it great again? and why? This statement is easy to agree with and hard to prove.

It also seems easy to handle links to different variants; just supply a URL parameter. This is a non-issue.

If someone out there has a non-tea leaf divination style alternative to measuring things as a way to determine success, I'm all ears, but I don't believe in fortune tellers, and people who make software shouldn't either.

	▲	liquicity 16 minutes ago \| parent [-]
		So now the average Joe has to make 3 vids instead of 1 every time - as everyone else is now min maxing their engagement? Just sounds rubbish both for viewers (look how terrible titles and thumbnails are after a/b tests) and uploaders alike