I'm referring to the failure of Hilbert's program. All the incompleteness, undefinability and undecidability results arise when and only when some sort of infinite objects are present so I can definitely see the allure of finitism.

ZFC is a working foundation of math but it's unknown whether it's consistent or arithmetically sound and important statements like CH are independent from it. It's a "working foundation" but not a "true foundation" which alas cannot exist.

As mentioned above I'm personally not a finitist and think that math without infinite and uncountable sets is intellectually poorer. I don't mind however developing further a finitist subset of math and see what's provable (and describable) in it, much like there's value in proving theorems in ZF instead of ZFC whenever possible.