> average songwriter isn't making data-driven decisions on how to settle on the chord structure for their song

Depends on what do you call data-driven. A songwriter most likely knows that a lot of fifth chords to gives power-metal vibes, and diminished and out-of-key songs do give these ghosts of jazz.