Its a nightmare because it erodes trust. Doctors are not "always right" which is why "always get a second opinion" is codified in culture.

But AI's problem is that its completely full of shit, sometimes, and the people most qualified to evaluate whether its full of shit are the doctors, not the patients, but just like OP's original article, patients are left feeling like their second opinion from AI might be more trustworthy than their doctors opinion.

The notion that only doctors can verify is false! Doctors are better at verification but normal people can also verify. This is just empirically true.

Examples of things normal people can verify

- procedural errors that Claude can capture like some blatantly high dosage (grams instead of milligrams)

- outdated treatment plan, maybe there’s a credible new treatment plan that’s been used for years but the doctors were not updated

- literally being injected homeopathic drugs (takes no smart person to flag this)

Let’s stop talking as if doctors have a divine right here. And let’s accept some agency.