What's also crucial is having your beliefs be backed by reality, and tossing away beliefs that aren't.

A core tenant of many-o-religions is faith, and if "the world" has counter-factuals, those are tests against that faith. Tossing away elements and breaking faith likely ends in exclusion from the group. And that is too much for many.