Because the verification would also be done by something non deterministic and then that’s a paradox.

Feed the LLM output into a “deterministic” verifier, problem solved. That’s how LLMs verify their new mathematical proofs with lean.