Here's a thought: once could derive the spectrum of the Markov transition matrix, and assign an entropy to each of the eigenvectors. The dominant eigenvector (highest entropy) would be the ergodic / stationary distribution, but it seems likely that each successive eigenvector would have a little less entropy. One could initialize the system in a "localized" state (very low entropy) and study the thermalization process as each of the low-entropy eigen-components decay away (exponentially, with rates proportional to the corresponding eigenvalue of the transition matrix) finally leaving the system in the high-entropy stationary distribution. The balance between the eigenvalues (exponential rates) and the entropies of respective eigenvectors would characterize the rate of entropy production (at different times) in the Markov chain!