And what's the most important aspect, in my opinion: In programming the LLM can make an hypothesis, try, get feedback, and refine the hypothesis, until it works.

This isn't the case in most areas. For example in Law, where everythign is text, you can RL so that an LLM produces an argument which a human would believe to be more reasonable, but you can't get a really fast loop of: make an argument, test it in front of a judge, refine the argument until you win the case.