"This has surprised scientists, and it remains an open research question to fully understand how our models produce such accurate predictions at this resolution."

Also crazy.

Seems important to understand why something does what it does, in the very least to know when it might not?

You can't, and it's one of the biggest problems when trying to use AI for anything.