Tao is continuously updating the cancer slide with the help of Claude in the past few hours.

https://github.com/teorth/tao-web/commits/main/

It reduces the badness somewhat.

"""An advanced AI is prompted: “Find a cure for cancer that passes a stage 3 clinical trial. Make no mistakes.” After a large amount of compute, it produces a cocktail of previously unknown chemicals which it claims, when mixed and injected into a patient, will kill all their cancer cells. While nobody truly knows how this cocktail was found, the AI (somehow) provides a Lean certificate for its prediction, and the cocktail does indeed manage to pass a stage 3 trial.

Could the AI solution be somehow misaligned by exploiting a weakness in the trial process or its math models?"""

Yes, absolutely an AI solution could exploit a weakness in the trial process or its math models. Would it remain uncaught? Unclear.