LLMs certainly make it more feasible to rewrite a product in a memory-safe language, eliminating a whole class of bugs.

Flawless software is hard for an LLM to write, because all the programs they have been trained on are flawed as well.

As a fun exercise, you could give a coding agent a hunk of non-trivial software (such as the Linux kernel, or postgresql, or whatever), and tell it over and over again: find a flaw in this, fix it. I'm pretty sure it won't ever tell you "now it's perfect" (and do this reproducibly).