LLMs certainly can interpolate between information found in its training set vastly faster than humans can find similar code on GitHub and stack overflow (or in math papers). It's certainly a tool that can be used when programming.
On the other hand also calculators help solve problems people have struggled with for millennia. It also doesn't mean that the end result is better than by hand. It's actually not, and that's the reason why you cannot replace programmers, because you need them to steer the LLMs for a good outcome.
And even the answer to the question, if LLMs make us more productive as programmers long-term is not really clear
