> it all rests on (relatively) simple mathematics. We know this is true. We also know that means it has limitations and can't actually reason information.

This was the statement I was responding to, it is stating that because it's built on simple mathematics it _cannot_ reason.

Yes we don't have a complete mathematical model of human intelligence, but the idea that because it's built on mathematics that we have modelled, that it cannot reason is nonsensical, unless you subscribe to a non-materialist view.

In a way, he is saying (not really but close) that if we did model human intelligence with complete fidelity, it would no longer be intelligence.

Any model we can create of human intelligence is also likely to be incomplete until we start making complete maps of peoples brains since we all develop differently and take different paths in life (and in that sense it's hard to generalize what human intelligence even is). I imagine at some point someone will come up with a definition of intelligence that inadvertently classifies people with dementia or CTE as mindless automatons.

It feels like a fool's errand to try and quantify intelligence in an exclusionary way. If we had a singular, widely accepted definition of intelligence, quantifying it would be standardized and uncontroversial, and yet we have spent millennia debating the subject. (We can't even agree on how to properly measure whether students actually learned something in school for the purposes of advancement to the next grade level, and that's a much smaller question than if something counts as intelligent.)