There’s really two types of people that tend to be drawn to libertarianism at some point: humanists and narcissists.

Those who are fundamentally humanist want to tear down systems of oppression because it pains them to see their fellow humans abused and brought low by corrupt laws and regulations. They (perhaps naively) imagine that if the system was dismantled or at least shrunk to minimum size, basic human decency will step in to fill the vacuum and people will thrive. Folks like Penn Gillette are the face of this group.

The narcissists are drawn to the movement because they feel like “if only everyone would get out of my way, I can do GREAT THINGS™ “. They like ideas like social Darwinism because they are already privileged enough to not be worried about losing in a survival of the fittest contest, and don’t tend to concern themselves with the second order effects of dismantling the system because it is simply an immoral impediment to their greatness. Peter Thiel and folks like him are the face of this group. This is largely the strain that has taken root in SV.

I see a whole lot more that end up at libertarianism as almost the default answer to "what philosophy most emphasis less government".

It's a generational thing I think, you see public money being spent on junk, and laws used to entrench and make competition hard; and you think "why do we want the government to do these things at all?". And if you look at common ideas around 20 years ago, the default answer was libertarianism.

fwiw Penn Gillette no longer calls himself a libertarian.