I think you are saying data is no substitute for vision in design. Completely agree! At Playdom (Disney) they tried to build a game once from the ground up based on A/B testing. Do you know what that game was? No you don't because it was never released and terrible.

I think what the previous comment meant was that there is data on how player play, and that tends to be varied but more predictable.

Yah. I think an AI playtester that could go "hey... this itch that lots of players seem to have doesn't get scratched often in your main gameplay loop" or "there's a valley 1/3rd of the way into the game where progression slows way down" or "that third boss is way too hard".

AI/fuzzers can't get far enough in games, yet, without a lot of help. But I think that's because we don't have models really well suited for them.