Replace AI with 'company' and you realize that we're living in that world already.

There's a reason it's called a Limited Liability Company. The whole point is to separate the company's liability from the individual that participates in it. For example, corporate financial liabilities of a company that is bankrupt.

You would not sue the nurse that makes a mistake. You would sue the company she works for. And if that company is bankrupt, well, that's it. There are rare instances where this isn't true, but generally it is.

Whether this is ethical or not, it's the system that we already have.

> AI can't be allowed to be a decision maker of any kind in healthcare.

It will, except it will be snuck in through the back door. It won't be some kind of robo-doctor making care decisions, it will come in the form of insurance eligibility steering the approved care plans, which absolutely will be decided by AI. The doctors will remain to act as liability sinks, but functionally they will probably join the ranks of the meat proxies.

Not to say that I don't agree with you to an extent, but you should see AI-delivered healthcare numbers to human-delivered and make it a fair comparison.

From Google:

"Annual Deaths: Over 250,000 people die each year in the U.S. due to medical errors made by healthcare professionals, including doctors and nurses. Ranking: This figure positions medical errors as the third leading cause of death in the United States, following heart disease and cancer."

It is highly possible that in a few years, AI will be a lot safer in the majority of healthcare situations.

> human lives are the cost of making the wrong decisions ...

True.

> ... AI can't be allowed to be a decision maker

Non sequitur.

Part 2 only follows from the Part 1 if you can demonstrate that AI makes more errors, or more severe errors, than humans in the identical situation.

And for those situations, I agree with you.

But there are already many situations where the AI is making fewer errors than humans, and for those situations, it would be unethical to exclude AI (see Part 1).

"A computer can never be held accountable; therefore, a computer must never make a management decision." - IBM, 1979

I'd say society's, or maybe executive's, viewpoint has shifted on this.

Why wouldn't they just blame the nurse anyway? The law always puts somebody on the hook. The nurse's job will be to do what the AI says and take the blame when it fails.

Excuse for who? The nurse? The hospital administration? The hospital owners? The insurance company?

There was quite a long debate about "Hallucination Defense" here: https://news.ycombinator.com/item?id=46815527

> “the AI did it" to be any acceptable excuse.

It will be depending on how much you paid. My bet is that in the near future normal people (like most of us) will be attended by AI in pretty much everywhere: the bank, the hospital, shops, hotels, etc.

You will need to pay extra premium to be attended by a human being. Actually that’s what’s been happening to the ultra rich since a very long time ago (e.g. a millionaire doesn’t book an appointment by using their phones, they ask their assistant to book for them; they don’t go online shopping, they get people do the shopping for them, etc)

I want to emphasize your point. As a basic matter of valuing humanity qua humanity, this holds even if the AI could be shown to make the decisions more accurately than a human could.

In fact, especially in those circumstances, if we take the warnings of X-risk even slightly seriously.

Obviously not an LLM, but AI models + robotics are likely to automate many healthcare tasks in the long term and are already beginning to automate tasks like blood draws.

  The Aletta autonomously handles each step of the blood draw, while a trained phlebotomist initiates each session and remains available to respond to any issues throughout. The device guides the patient to position their arm, after which the patient or supervisor presses a button to start the blood draw. The Aletta then uses near-infrared light and Doppler ultrasound to locate a suitable vein and tell it apart from arteries. If no appropriate vein is found, the device will not attempt the procedure.

  Once a vein is identified, the Aletta proceeds through the remaining steps on its own: applying a tourniquet, preparing the skin, inserting and disposing of the needle, changing collection tubes and placing a bandage. The supervising phlebotomist confirms that collection tubes are filled in the correct order and verifies that all tubes are adequately full following the procedure.

Yep, I read it, that’s what this seems to be about

It feels like propoganda.

Never underestimate the ability of people in tech to misunderstand the real world. I'm sure there's a couple dozen xkcds about this.

There's an optimistic future where Medicare becomes so much more efficient it becomes politically obvious to expand it because it's outperforming the mess that is private sector healthcare.

With AI I could actually see this being a possibility since it doesn't depend on a huge army of competent employees in a government office.

Marshall Brain really nailed it back in 2003 with Manna. https://marshallbrain.com/manna1

AI leaves humans as the limbs of the machine, with less and less thinking or comprehension required on their part.

No? Prices are market-driven in our society, not necessarily a function of labor costs. In the very long-term best case scenario, the increased profits going to the company from these efficiency gains may encourage more competitors to pop up, and either drive prices down or offer more for what you pay.

> theoretically

In case you haven't noticed, all that our current system is able to do with increased efficiency is cutting staff and/or increasing the percentage of money that gets siphoned up to capital holders.

Healthcare prices are not dictated by labor costs, but by how much the leeches who own everything are willing to charge people for it. For example, insulin is dirt cheap to make, yet billed at exorbitant amounts.

It will always be until we have unlimited resources.

[dead]

The industries with the highest costs - health care, housing, education - are the industries most heavily interfered with by the government.

The industry with an absence of government interference, software, has seen prices drop to zero.

youre comment is getting to the real economic future: nursing unions will not sit back and let this happen. if the "productivity fantasies" (well said) are real this will get ugly. the competent nurses where i live are in strong unions

Invisible hands are the tool of pickpockets.

Air travel is a good counterexample here: technology made airlines far more productive, but much of the gain went into lower staffing, higher throughput, and cost cutting not a better passenger experience

True, but you ignore the financialization and enshittification that typically follows the technological advances. The tech processes you cite don't inherently make money for the people who developed the technology.

During the industrial revolution, people thought that all these new machines and factories and electricity meant that everyone would only have to work 4 hours a week since it made everyone so much more productive.

That's not what happened, though. Everyone worked the same amount for the same pay, just in more miserable repetitive working conditions, while the factory owners kept all profits from the gains in productivity.

In a market with perfect competition, perhaps. Healthcare is far closer to a monopoly. Since people can't decide not to be sick, it means you can pocket the cost-saving.

>And outside of industries with government enforced perversions of simple "supply and demand" forces (we're not talking Nth level effects here) this means more access to the service or lower prices or both.

Greater access at the expense of the number of people employed is not necessarily a net positive. In the case of healthcare, the data is fairly clear that higher nurse staffing ratios lead to better patient outcomes (read: decreased chance of death or serious complications). This is particularly true for ED and ICU nurses.

Also, in the case of healthcare, I'm fairly certain that hospitals in the US are legally not allowed to turn patients away. So "lack of access" is not currently a problem for serious medical emergencies.