> I genuinely don't understand why people don't get more upset over hitting refresh on a webpage and ending up in a significantly different place.

I'm in the opposite camp - I find it extremely annoying when sites clutter up the browser history with unnecesarly granular state. E.g. hitting "back" button closes a modal instead of taking me to the previous page.

You can achieve both a clean history and granular state in the URL with using history.replace() and history.push() where necessary.

I think that'd be too much. A modal is a subordinate thing to the current window, so I think it shouldn't merit a full url change by itself...

This is a completely different issue; you can replace history state in JS without adding new entries.