I think the issue people run into is that the types of errors the LLM makes are ones humans wouldn’t make, like having a basic misunderstanding of the goal of the software and making silly logic errors that technically work, but don’t serve the correct purpose.

This can be seen more clearly with self-driving cars as an example. A self-driving car may be safer than a human driver, but when the self-driving car plows into the side of a semi truck in broad daylight… that’s generally not a mistake a human would make. Humans and AI have different failure modes, so when AI fails where we generally wouldn’t, it really stands out and gets judged harshly.