I disagree! It's easy to check that an AI program meets its specification, which is to process input tokens and generate output tokens. :)

If you're talking about verifying whether it produces the correct tokens, that's not generally something you can specify in advance with AI. I mean: if your task is one where you can precisely specify which output tokens are correct for a given input, then the task doesn't need AI, no?