Yes but many of the challenges directly contradict the idea that "coding is a solved problem"

Not really.

Coding is probably solved, at least to a large extent, but that doesn't mean engineering is solved too.

This is like someone saying that the wright brothers solved sustained/powered human flight, and another person saying "well if that's the case, why do planes still crash? obviously flight isn't solved.". Well, there are always improvements, but planes can fly and llm's can code.