> but unless someone--a human--can verify and understand it

Why though a human? Mathematical proofs are very ivory-towery, but if OpenAI would solve a subkind of Cancer, without mortal humans understanding, cancer is still be healed.

Not unless humans were able to produce the medicine or perform the surgery or whatever is necessary to actually heal someone. In order for that to happen it has to be verified and understood by humans, like the other commenter said.