I think he meant that as "I live in the UK where this is already bad, yet the EU still ended up worse.".
This is how I also read it.