Yes NLP has come a long way and LLMs allow easier "searching" of the solution.

The one shot results vary in quality because in order for the result to align with something good, you're playing a guessing game which description(s) match the desired output. Needing to know what's on both sides of the curtain is in some ways potentially double the effort of not bothering with an LLM in the first place.

It's also for this reason I'm skeptical of the more "specialized" models. They're basically inflatable bumpers on the bowling lanes mapping to limited outputs that might not even be as good as the training source. Might as well just give me the github link at that point and we're right back to copypasta in the 2010s.