Can this be solved by setting p=0.50?

Make your expectations explicit instead of implicit. 0.05 is completely arbitrary. If you are comfortable with a 50/50 chance of being right, make your threshold less rigorous.

▲

scott_w 11 hours ago | parent [-]

I think at that point you may as well skip the test and just make the change you clearly want to make!

▲

bigfudge 10 hours ago | parent [-]

Or collect some data and see if the net effect is positive. It’s possibly worth collecting some data though to rule out negative effects?

	▲	scott_w 8 hours ago \| parent [-]
		Absolutely, you can still analyse the outcomes and try to draw conclusions. This is true even for A/B testing.