> The assumption I need to challenge is that "an ultraintelligent machine could design even better machines".

Would you bet your life, and everyone else's, to test that assumption?

Not to mention: you can scale up performance in simple ways: more data, more compute, and more weights. Those are the obvious ones, ignoring architectural changes.