Inconvenient, most of the times freaking expensive or vendor locked, no privacy - these are major reasons why it doesn't work out.

A good VR headset will be most probably a form of wireless screen (not a beafy computer) that is simple, lightweight (at most 200g in front of your face), cheap enough so you'll be able to wear it with you and not worry that something happens to it or be able to afford it for each member of your family without major effort and most of all - it should be replacing computer/laptop screen for daily tasks.

We could do it today, big tech is just holding us back with their vision of products