I know this is just an example, but I'm going to say this every time the subject comes up: since human lives are the cost of making the wrong decisions, AI can't be allowed to be a decision maker of any kind in healthcare.

If my blood pressure crashes because my nurse is caring for 15 patients at once and didn't notice that the AI decided to pause my norepinephrine, I won't consider "the AI did it" to be any acceptable excuse.

> AI can't be allowed to be a decision maker of any kind in healthcare.

It will, except it will be snuck in through the back door. It won't be some kind of robo-doctor making care decisions, it will come in the form of insurance eligibility steering the approved care plans, which absolutely will be decided by AI. The doctors will remain to act as liability sinks, but functionally they will probably join the ranks of the meat proxies.

Not to say that I don't agree with you to an extent, but you should see AI-delivered healthcare numbers to human-delivered and make it a fair comparison.

From Google:

"Annual Deaths: Over 250,000 people die each year in the U.S. due to medical errors made by healthcare professionals, including doctors and nurses. Ranking: This figure positions medical errors as the third leading cause of death in the United States, following heart disease and cancer."

It is highly possible that in a few years, AI will be a lot safer in the majority of healthcare situations.

> human lives are the cost of making the wrong decisions ...

True.

> ... AI can't be allowed to be a decision maker

Non sequitur.

Part 2 only follows from the Part 1 if you can demonstrate that AI makes more errors, or more severe errors, than humans in the identical situation.

And for those situations, I agree with you.

But there are already many situations where the AI is making fewer errors than humans, and for those situations, it would be unethical to exclude AI (see Part 1).

Excuse for who? The nurse? The hospital administration? The hospital owners? The insurance company?

"A computer can never be held accountable; therefore, a computer must never make a management decision." - IBM, 1979

I'd say society's, or maybe executive's, viewpoint has shifted on this.

Why wouldn't they just blame the nurse anyway? The law always puts somebody on the hook. The nurse's job will be to do what the AI says and take the blame when it fails.

There was quite a long debate about "Hallucination Defense" here: https://news.ycombinator.com/item?id=46815527

> “the AI did it" to be any acceptable excuse.

It will be depending on how much you paid. My bet is that in the near future normal people (like most of us) will be attended by AI in pretty much everywhere: the bank, the hospital, shops, hotels, etc.

You will need to pay extra premium to be attended by a human being. Actually that’s what’s been happening to the ultra rich since a very long time ago (e.g. a millionaire doesn’t book an appointment by using their phones, they ask their assistant to book for them; they don’t go online shopping, they get people do the shopping for them, etc)

I want to emphasize your point. As a basic matter of valuing humanity qua humanity, this holds even if the AI could be shown to make the decisions more accurately than a human could.

In fact, especially in those circumstances, if we take the warnings of X-risk even slightly seriously.