If it’s anything like the AI expert systems I’ve heard about in insurance, it will be a tool that is optimized for low effort, but will be used carelessly by end users, which isn’t necessary the fault of the AI. In automated insurance claims adjustment, the AI writes a report to justify appealing patient care already approved by a human doctor that has already seen the patient in question, and then an actual human doctor working for the insurance company clicks an appeal button, after reviewing the AI output one would hope.

AI systems with a human in the loop are supposed to keep the AI and the decisions accountable, but it seems like it’s more of an accountability dodge, so that each party can blame the other with no one party actually bearing any responsibility because there is no penalty for failure or error to the system or its operators.

▲

reactordev 2 days ago | parent [-]

>actual human doctor working for the insurance company clicks an appeal button, after reviewing the AI output one would hope.

Nope. AI gets to make the decision to deny. It’s crazy. I’ve seen it first hand…

	▲	aspenmayer 2 days ago \| parent [-]
		It gets worse: I have done tech support for clinics and a common problem is that their computers get hacked because they are usually small private practices who don’t know what they don’t know served by independent or small MSPs who don’t know what they don’t know. And then they somehow get their EMR backdoored, and then fake real prescriptions start really getting filled. It’s so much larger and worse than it appears on a surface level. Until they get audited, they likely don’t even know, and once they get audited, solo operators risk losing their license to practice medicine and their malpractice insurance rates become even more unaffordable, but until it gets that bad, everyone is making enough money with minimal risk to care too much about problems they don’t already know about. Everything is already compromised and the compromise has already been priced in. Doctors of all people should know that just because you don’t know about it or ignore it once you do, the problem isn’t going away or getting better on its own.