As a libertarian whose thinking has gone through the whole spectrum of left-right and back again, the fundamental problem is that [pure] right libertarianism is inherently contradictory, despite its simplicity making it extremely attractive. A core principle of rightist thinking is an assumption that there is some bedrock of moral axioms, and as long as we follow them then the resulting situations must also be morally right by construction. This directly clashes with Gödel's famous results in logic, which show that complexity itself creates new logical contradictions. Rightest libertarians (eg the bulk of the Libertarian party, Constitutional fundamentalists, etc) are still running off the failed ideas from the 1910's that produced efforts like the Principia Mathematica.

The way I've come to see it, left and right essentially correspond to two modes of reasoning, inductive versus deductive - they are both required to get anywhere worthwhile. The current highly divisive political environment is essentially making everybody think with only half their brains. This is both lucrative (it feels good to have lazy answers validated rather than criticized), as well as disempowering (it keeps individuals from agreeing on substantive political opposition to ever-growing corporate authoritarianism).