Is going public the ultimate goal of every startup?

The goal of the startup doesn’t matter once they take VC funding. The goal of the investors is the exit - either via acquisition or going public.

The most likely outcome is failure, the second most likely outcome is an acquisition. Going public is a distant third