I used to read a lot of LessWrong, Yudkowsky, SlateStarCodex. Rationalism is really just another religion, with the faith parts carefully tucked away underneath concepts like priors or how one chooses to construct a model or assign/“update” probabilities. Their god is ASI. Their beliefs have no more fundamental basis in truth than any other faith.

There’s also a fundamental problem with the idea of using models and probabilities for decision making in that one has to choose a model before deducing any facts, and this leads to all sorts of post hoc rationalization. It’s basically just a culture of people with weird preferences coming up with mathy rationalizations for them.

It seems unfair to conflate the rationalism of Berkeley, Locke, Russell and Hitchens who hold critical thinking in the highest regard, to AI maximalists, who seem to be using rationalidm mainly as a cover for their own pet ideas on social reengineering.

The former is mainly about being true to the scientific process of evidence-led discovery, while the latter is... whatever it is (but rational it is not). From a critical thinking perspective, there is no evidence to suggest the current form of AI could achieve superintelligence, and mechanisms that could enable ASI have not been discovered yet. Their perspective, on the other hand, is far more activist.