I'm not clear why you are focusing on hashing user ids. Nor how you landed on a 50/50 split. When someone logs in, write a record of which one they got. No need for hashing. The algorithm for choosing can be any percentage. And yes, just a simple lookup table is fine.

But the work comes when analyzing the data later, so that is where you need to put in more thought. What are you measuring? How are you logging the events that will track that measurement? Those are both harder and more important questions. Based on what you are measuring, you might want only specific subsets of users to test things - why assign 50% of an entire userbase to a feature being tested that only 10% of the users touch? You need to assign a portion of that 10%, not a global 50%.

Basically, you need to start with the end data you intend to act on and work backwards, not start with a hashing algorithm that might not even help.

Thank you for helping!

> I'm not clear why you are focusing on hashing user ids. Nor how you landed on a 50/50 split

I landed on hashing and splitting from my research on building A/B tools, but none of that research was targeted towards building real, enterprise products (which is why I asked the question here). From your reply, I take it that this isn't as important as I read about earlier?

> When someone logs in, write a record of which one they got.

I'm confused about what you mean by "which one they got". How do I know which version to assign them? This is what I assumed hashing would solve - we'd have a reliable way to "choose" a version for any given user.

> why assign 50% of an entire userbase to a feature being tested that only 10% of the users touch?

This makes sense, I'm not sure why I had landed on 50%. So the percentage difference does not matter? I had assumed that we need a way to enforce a certain percentage split - how do I prevent a "feature" from only reaching 0.01% of the userbase, whereas the other feature reaches 99.99%?

Thanks again for your reply, you've been really helpful.