> But there is no mechanism to generate a model with capabilities beyond what is useful to minimize a specific cost function.

Can you give some examples?

It is not trivial that not everything can be written as an optimization problem.

Even at the time advanced generalizations such as complex numbers can be said to optimize something, e.g. the number of mathematical symbols you need to do certain proofs, etc.

I think you're misreading me. My point isn't that you can't in principle state the optimization problem, but that it's much easier in some domains than in others, that this tracks with how AI has been progressing, and that progress in one area doesn't automatically mean progress in another, because current AI cost functions are less general than the cost functions that humans are working with in the world.