Isn't the EU rather like a single point of failure?

No, every country remains sovereign. Hungary's previous regime ignored the EU for over a decade. Many countries are instituting border controls again despite the Schengen agreement.