Is it really?

I think the assumption that there is some sort of formal ‘reasoning’ process that is inherently superior to whatever ‘non-reason-based’ decision process ‘non-reasoning’ systems use is not immediately justifiable.

Most reasoning is ultimately based in heuristics: this is the right thing to do next because it has worked in the past. This is the right thing to do because our intuition tells us doing it will lead to this desirable outcome.

When you’re formulating a math proof, sure, the steps you are laying out make up a ‘reasoned’ argument but how you choose what steps to take is wholly intuitive. Why assume the contrapositive to begin this proof? It’s worked in the past and it might work here.

It’s all pattern matching trained on a reward function.