Right, it also has to model a substantial fraction of reality (or at least a true simulation of it) to accomplish these things.

It models our language, which is a flawed and imperfect way of describing the world. So far, it seems like a lot of these discoveries are "filling in the gaps" between the things we've written down and the things they imply (if you have the memory to think them through).

The story about OpenAI's Navier-Stokes solution is a good example of what I mean. I don't think it would have been possible without computer assistance because that proof is long and complicated. I'm also not sure that it would have been possible without a human proposing a new approach to the problem, because by all accounts that's exactly what led to the absurd amount of spending that OpenAI did to solve the issue.

I feel like that at least implies that there's some room left for humans in the new world.