I think the idea is to just have the LLMs write the control systems and RL on that

Sure hope they're not writing the safety cutoffs as well. Some of those systems do not take kindly to bad control inputs, and determining if a controlled system is stable can be absurdly hard.