Neural networks are already systems of linear algebra that are beyond human understanding. Most humans could probably grok a 1 or 2 dimensional slice of a network, but the latent vector space is completely beyond the human brain. We have to use tools to analyze neural networks piecemeal in exactly the same way that we analyze any other higher-dimensional construct. Few humans are truly capable of reasoning in 4+ dimensions, that doesn't make string theory "not math". Nor does a trillion-dimension vector space of an LLM make it "not programming".

Humans by themselves invented mathematical concepts beyond human understanding a long time before we invented neural networks.