Not quite. Large amounts of data going into these models has already been curated, otherwise you would get a tremendous amount of wrong answers for even the most basic questions.

It still produces wrong answers regardless though, not because of the training data but because of just... intrinsics. The question is what an acceptable error rate is, how severe those errors are, and whether a human would make comparable errors.

But this debate has parallels with self-driving cars; even if the numbers say that self-driving cars are not perfect but safer than human drivers, anything but perfection will be considered broken or outright illegal.