For most mathematicians, most of the time, the mathematics they personally do is, in relation to different mathematical foundations, like architecture is to atoms.

Architects know and care that they are building things made of atoms. And then ... pretty much don't think about atoms because the objects and relationships they are working on are abstractions well above the fine details of atoms.

And having designed many structures with architectural methods, and seeing those buildings built and stand, it doesn't worry them to hear physicists arguing that maybe atoms are different than they thought. They figure, their experience with architecture and its artifacts has proven to be reliable, so there is no realistic threat of some new quantum theory undermining their work.

On infrequent situations, where their work needs to deal with some special property of some material, they don't have any issue dipping down any number of levels of abstraction. But as a practical matter, that is infrequent for most math.

Thanks, this is a very good analogy.