If your site is only hosted on one server and it catches fire, you can swiftly reinstall on a new server and change the IP your domain is pointing to, too... Still a single point of failure.

Yes, everything in the world is a single point of failure and has always been, if we look at things that way. But if it can be remedied quickly, then it's not a huge concern.