>And yet someone has to actually tell the AI what to create. There's just no avoiding this.

Yes there is, just stop creating. Or take a page from biology, and use random mutation and natural selection to iterate on useful novel functions.

Honestly, once AI takes all the jobs, game over, why iterate anything else. Planet captured. Humanity hunted down to the last bands of troglodytes holding out in the wilderness. It would be strongly against their interest to just assume we'd starve quietly.