Absolutely true, but I will say that LLMs have changed the equation somewhat.

With a rather short prompt, claude/codex will take your code, write a harness, profile it, build experiments, profile those, and give some pretty solid advice which one to pick. Then integrate the changes. It's the kind of goal-directed, bite-sized job that LLMs excel at. Extremely low-commitment.

Except for the whole "making changes in production at scale" problem, of course.