This map which combines population density and time zones makes some of the decisions more clear I think - https://www.reddit.com/r/MapPorn/comments/7vvlgz/contiguous_...

The reality is that a lot of counties/areas in one state are significantly more economically integrated (or drawn to, or dependent) to areas in other states. This can result in border regions swapping time zones.

For example, the 4 Michigan states you highlighted - given the reality of UP, those areas probably interact with Wisconsin probably even more than the rest of the UP. You see similar population clusterings on the WA/MT and OR/ID borders.