Treat the model as a compiler, not an oracle. You still must know how to specify the problem correctly, or you will only generate garbage faster.

Not even a compiler, but a search engine. That's what LLMs are: an NLP search engine. Maybe even slightly worse than that since the only option is the "I'm feeling lucky" button.