> One way that thinking for yourself goes wrong is that you realize your society is wrong about something, don’t realize that you can’t outperform it, and wind up even wronger.

many such cases

It is an unfortunate reality of our existence that sometimes Chesterton actually did build that fence for a good reason, a good reason that's still here.

(One of my favorite TED talks was about a failed experiment in introducing traditional Western agriculture to a people in Zambia. It turns out when you concentrate too much food in one place, the hippos come and eat it all and people can't actually out-fight hippos in large numbers. In hindsight, the people running the program should have asked how likely it was that folks in a region that had exposure to other people's agriculture for thousands of years, hadn't ever, you know... tried it. https://www.ted.com/talks/ernesto_sirolli_want_to_help_someo...)

Capital-R Rationalism also encourages you to think you can outperform it, by being smart and reasoning from first principles. That was the idea behind MetaMed, founded by LessWronger Michael Vassar - that being trained in rationalism made you better at medical research and consulting than medical school or clinical experience. Fortunately they went out of business before racking up a body count.

One lesson I've learned and seen a lot in my life is that understanding that something is wrong or what's wrong about it, and being able to come up with a better solution are distinct, and the latter is often much harder. It seems often that those that are best able to describe the problem often don't overlap much with those that can figure out how to solve, even though they think they can.

It's almost the defining characteristic of our time.

indeed

see: bitcoin