At this point I think the category theorists hit the foundational idea squarely on the mark:

1. Start with a few simple but non-trivial terms and axioms

2. Define "universal constructions" as procedures for building uniquely identifiable structures on top of that substrate

3. Prove that various assemblages of these universal constructions satisfy the axioms of the substrate itself

4. "Lift" every theorem proven from the substrate alone into the more sophisticated construction

I'm not a mathematician (I just play one at my job) so the language I've used is probably imprecise but close enough.

It may be true that you can't prove the axioms of a system from within the system itself, but that just means that you need to make sure you start from a minimal set of axioms that, in some sense, simply says "this is what it means to exist and to interact with other things that exist". Axioms that merely give you enough to do any kind of mathematics in the first place, that is. If those axioms allow you to cleanly "bootstrap" your way to higher and higher levels up the tower of abstraction by mapping complex things back on to the simple axiomatic things, then you have an "open" or infinitely extensible system.