Reading this, I'm reminded of... New Coke.

Despite post facto conspiracy theories that it was a guerilla marketing campaign to promote "Old Coke", it was not. Coca-Cola tested a new formula in focus groups and people preferred it at the time because, basically, it was sweeter. But this didn't translate into longer term consumption. It's a cautionary tale in how data science can fail and how you shouldn't necessarily extrapolate to the long-term from the short-term.

So when people say they preferred AI written stories, what are they really reacting to? There's probably a lot of elements that are engaging but is it a lasting story? Most enduring stories become so because they speak to some truth of the current times or the human condition that can be hard to separate from the technical merits of the story.

Example #1: Fight Club both as a book and a movie was a bit of a sleeper hit. It was a box office failure. But it became a cult classic because it was one of the earlier works that spoke to the loss of a future for men in our society.

Example #2: The Rocky Horror Picture Show. Again, it was a sleeper hit but it built over time as a relatively early Western expression of queer culture, gender identity and cultural conformance.

Example #3: The Wire. While it had great writing it has endured as a testament to institutional failure at every level and that doesn't necessarily take a villain. Everybody can be acting in their own interests and you end up with inevitable failure anyway.

Example #4: The Great Gatsby. This was never a hit in F. Scott Fizgerald's lifetime. How it became one is an interesting story. Basically, during WW2 the Army created these small print books for deployed soldiers to easily carry and read. Then once the war ended and many of the veterans went to college, they wrote about the Great Gatasby. It's now viewed as an American masterpiece that captures the 1920s like few other works. It expresses class consciousness and some of the metaphors (like the green light on the dock) are debated to this day.

Now, could AI write things that coherently speak to the human condition this way? Not yet (IMHO). Far from it.

That being said, there are only so many base story structures. Take the romance category. You'll find repeated tropes. Could AI write what might best be called "pulp fiction" in that genre? Probably sooner rather than later.