I spent five years working in quality assurance in the manufacturing industry. Both on the plant floor and in labs, and the other user is largely correct in the spirit of their message. You are right that it's not just up to things being easy to spot, but that's why there are multiple layers of QA in manufacturing. It's far more intensive than even traditional software QA.

You are performing manual validation of outputs multiple times before manufacturing runs, and performing manual checks every 0.5-2 hours throughout the run. QA then performs their own checks every two hours, including validation that line operators have been performing their checks as required. This is in addition to line staff who have their eyes on the product to catch obvious issues as they process them.

Any defect that is found marks all product palleted since the last successful check as suspect. Suspect product is then subjected to distributed sampling to gauge the potential scope of the defect. If the defect appears to be present in that palleted product AND distributed throughout, it all gets marked for rework.

This is all done when making a single SKU.

In the case of AI, let's say AI programming, not only are we not performing this level of oversight and validation on that output, but the output isn't even the same SKU! It's making a new one-of-a-kind SKU every time, without the pre and post quality checks common in manufacturing.

AI proponents follow a methodology of not checking at all (i.e. spec-driven development) or only sampling every tenth, twentieth, or hundredth SKU rolling off the analogous assembly line.

In the case of AI, it gets even worse when you factor in MCPs - which, to continue your analogy, is like letting random people walk into the factory and adjust the machine parameters at will.

But people won't care until a major correction happens. My guess is that we'll see a string of AI-enabled script kiddies piecing together massive hacks that leak embarrassing or incriminating information (think Celebgate-scale incidents). The attack surface is just so massive - there's never been a better time to be a hacker.