We used to carry paper pieces called 'bank notes' or 'bills' and round metal disks called 'coins' in a small leather pouch in our pockets called wallets. They were pretty effective for payments without much of an infrastructure. Even banks worked using paper documents and books.
I know this sounds a bit too condescending. But that's honestly not my intention. I just couldn't help it! Jokes aside, it's true that we often forget that these things can be done and were done without the internet. But more importantly, there are 2 dangerous implications for our over reliance on the internet for our financial activities. The first is that the government or a non-state actor can easily disrupt our commercial and personal activity unintentionally or as a retribution. We have effectively surrendered our financial autonomy to multiple powerful players.
The second major problem is if we ever face a post-apocalyptic situation with regards to modern technology. We already have only a few fabs that can meet the global demand for advanced ICs. We have already seen our vulnerability to one of them when a flood there caused supply chain disruptions and a slump in even automobile markets. HDD and SSD manufacturers have similar weaknesses. Meanwhile, DRAM manufacturers are placing all their (gambling) chips in the AI hyperscaler market, threatening to disrupt every market from smartphones, laptops and consumer appliances to military and commercial jets, ATC, shipping, railway signalling, telecom infrastructure, etc. The technology apocalypse isn't that farfetched and we are extremely vulnerable to it.