>> I think that over-reliance on analytics has had a similar effect

I lost track of how many times the directors would tell us, "Since the analytics say A, we should do B." then the research person says, "Sure Jim, great idea, lets get a few rounds of useability research and confirm it first."

Then the always predictable thing happens: Users never align with your analytics. Seeing data and seeing someone struggling to do something basic with your interface is totally different. When directors see these videos, it really makes them see how important research is and not just relying on data to make decisions. Right now, anything the research team wants, they usually get - its had that profound of an impact on our leadership team.

▲

asdfman123 36 minutes ago | parent | next [-]

My problem is I can skip all those steps and 8 separate meetings by applying a little common sense to software design.

Unfortunately, orgs often treat devs like this as indulgent or wasting time. They need it to be done the inefficient way because that's the only way they have visibility and control.

▲

zahlman 2 hours ago | parent | prev [-]

> Users never align with your analytics.

Any good theories for how this happens (i.e. why the data fails to capture the struggle)?

	▲	layer8 2 hours ago \| parent \| next [-]
		Analytics don’t capture the “why” of what they measure, on the user side. Indicators may move in the seemingly right direction for the wrong reasons. Analytics typically can’t tell you what the user wanted to achieve. Knowing the “why” gives a better basis to decide on what changes to try, or to realize what’s actually wrong with the user-facing side of things. Another reason is that they tend to measure an average where in reality there is no average user.
	▲	zmgsabst 8 minutes ago \| parent \| prev \| next [-]
		Interpreting data is hard. Eg, if you have a bunch of airplanes returning from a war, you might be tempted to armor the areas where they were hit. But you want to armor planes where the holes arent because those are the critical areas. If you can read literal bullet holes backwards, hundred dimensional preference vectors say something, but we generally have no clue.
	▲	skydhash 2 hours ago \| parent \| prev [-]
		Not GP, but I can humbly share mine. I think it's because a lot of analytics are data points oriented instead of being workflow oriented. So you can see that feature A is not being used a lot, but it's very important in a particular flow. Feature B may be used a lot, but it can be only important for a particular class of users while very detrimental mentally for another class. People use software for a needs, but rarely I've seen a need being highlighted when interpreting analytics data.