That can only be true for a definition of 'elite' that excludes the Eton and Oxford educated Prime Minister of the day and the majority of his government. So I mean, yeah, some people wanted Brexit. If those people had some sort of sinister behind-the-scenes power to bring it about, you have to wonder why Brexit only actually happened after the referendum vote and not in the decades prior, during which those very same people had been continually agitating for it.

>the elite only allowed the vote because they knew how it would go

Clearly not, in this case. Remain was forecasted to win, and David Cameron called the referendum because he felt confident that remain would win.

>they would've brexited even if remain won

This also is obviously not the case. David Cameron would have remained Prime Minister. He was always in favor of EU membership, remaining in the EU was part of the policy platform of his party, and he would have been under much less pressure from Brexiteers than he was before the referendum.

>What is not possible is that the elites wanted to remain but were forced to leave by the vote

This is in fact what happened. David Cameron resigned after losing the vote and was replaced by someone committed to implementing Brexit.