A way I've been developing and is working really well is to identify high level "goals" of the day or week by analyzing their AI chats first.

Then, measure time taken, AI usage, and sentiment of AI usage.

With this, we find out how quickly was the task done, how much AI was used, and whether the individual was frustrated at any point and if the process went smoothly etc.

My system already hooks into top AI providers and measures these outcomes for engineers. Working on measuring other use cases. Email in profile if anyone wants to chat.

Now of course we can't do a blind comparison with the exact same task, but this at least gives insights into usage, outcomes, and ease-of-use.