There is zero evidence that LLMs can do anything novel without a human in the loop. At most LLM is a hammer. Not exactly useless by any stretch of the imagination, but yes you need a human to swing it.