>That the rich will flee, leaving us.... better off in every possible way??

Can you explain why would we be better off when the rich flee? I see this "we gonna be better off without them" sentiment quite often, but never an explanation how would we be better off? What metrics have you used?