This is exactly the answer, and it's why so-called "libertarianism" is not just naive but destructive (and end-result authoritarian) as an ideology: it has a fundamental misunderstanding of what the "free market" actually is behind the scenes, imagining it as some sort of neutral/neutralizing / democratic mysterious force that prevents accumulation of power and being in opposition to "the State" when it is the opposite. "Free market" systems create the State and then dominate it and the interests involved create power systems for their own accumulation of dominance. And the end result is oligarchy.

The counterveiling force has to be an organized working class that uses unions, political parties, and democratic force to take back at least some control of the state and market and force its use for social means. And this is a constant battle. If we start to lose, which has happened progressively since the 70s, Trumpist oligarchy is the result. Or worse.