We can model physics and chemistry pretty well, just not beyond small scales, because the computational effort blows up.

You could just as easily say that if you can write a python interpreter that you can understand every program written in python. Ok, now what if the program is two terabytes?

A frontier LLM is nothing but a 2 terabyte program written in a weird programming language. Just because you can understand the interpreter does not mean you understand the program in a meaningful way.