Discovery and optimization are very different processes. Optimization is the process of finding the shortest path to a goal. Discovery is the process of stumbling on new goals and redrawing the map of what's possible.

Ambitious goals and new discoveries happen via novelty-based search. Progress in scientific discovery is measured by how different/interesting the outcomes are, not by closeness to a predetermined goal.

Discovery is a creative search that preserves optionality, whereas optimization restricts optionality. In other words, you usually don't discover anything novel unless you're trying new things that don't appear connected to the goal in the first place. Would an ML optimization loop have discovered transformers?