I RFTA and the majority of the complaints are about call center metrics and the pressure to ration care. These are real concerns about misuse of metrics, but not AI. The AI empathy thing was a 2024 pilot that was discontinued.

FWIW my wife works for Kaiser and finds a lot of value in the the medical LLM tools available to her. She tells me being able to do live translation, summarize notes, and quickly get comprehensive answers save her time and help her give better care. Her older patients also frequently come in bringing AI-powered alerts from their apple watches that detected cardiac events.

It's annoying that we use broad terms to describe a set of technologies that in some ways can be problematic and in another ways are very beneficial. We gotta evaluate each of these as they come rather than talk about blanket bans.

Being close relative for several med and care workers we have discussed it a lot and consensus is that it really depends. For example relying on LLM summaries sounds great until it doesn't. It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize.

Relying on LLM to summarize things for you has one more issue. To outsiders, this seems like a tedious process, but is actually very important part of the thought process. Wording your thoughts and writing these down helps people to discover new aspects of the problem. It's how people learn.

At the moment consensus is that it must not be banned, but also not mandated in any way - people must take responsibility, and they must be able to decide for themselves where and when the LLM use is justified and where it is not.

Things like this are (sadly) common (and age-old) problems with automation and computerization. (For a vivid account of this phenomenon, check out the novel _Close to the Machine_, by Ellen Ullman.)

As executives and analysts increasingly use the "AI" craze to push automation and computerization (and layoffs) generally, even aside from AI proper, it should not be surprising that the individuals and groups opposing those moves also use the same labels.

The lack of precision in language here sucks. It sucks for the discourse and it also sucks when it comes to focusing anger and productive energy on the core problems (obfuscation of human responsibility, erosion of human agency, declining institutional flexibility, deprofessionalization, etc.). But it doesn't begin with the critics of AI.

> AI-powered alerts from their apple watches that detected cardiac events

Surely these are “good old-fashioned AI” (statistical learning) and not LLM, though.

I just want to be clear that the “medical LLM” tools are the new ones, and the Apple Watch alerts aren’t.

Cardiac events from Apple Watches is not “AI” though

Just for clarification, is your wife a doctor or a nurse?

>> It's annoying that we use broad terms to describe a set of technologies that in some ways can be problematic and in another ways are very beneficial. We gotta evaluate each of these as they come rather than talk about blanket bans.

I totally get it. I think few years, if some company said they record and transcribe every meeting/interview they take, it would be concerning. Now, its somewhat a norm for people to use these AI meeting tools which record everything you say and then go back to recording and exactly what people said. I'd call it surveillance than AI

everyone is very thirsty for AI hate, so its not unexpected. Today its a mixed bag of corpo hate, anxiety about the future, inequality and traditional class warfare, combined with the typical technical ignorance.

I would expect companies to blend shit metrics with AI systems, if not at Kaiser then at other places. People lack imagination and using AI to monitor your workforce has to be one of the possibly worst ways to use it. Alternatively some dickhead will "lean startup" their way into measuring "performance" in such a way with the "help" of AI that they will do something even worse.

“There is always a point at which the Amodei ceases to manipulate the media gestalt. A point at which the capital misallocation may well escalate, but beyond which the Amodei has become symptomatic of the capital markets themselves. Amodei as we ordinarily understand it is innately media-related. The Hangzhou hackers differ from other AI engineers precisely in their degree of self-consciousness, in their awareness of the extent to which media divorce the act of giving Amodei fuckin money from the original sociopolitical intent.”

[dead]

I personally will not use a provider who uses llm tools. I know it makes me and my coworkers less careful and lazier. Qualities I dont want in a health care providor.

Ive actually moved primary care physicians over this once already, found the oldest guy I could who barely knows how to use a laptop but spends a bunch of extra time with me.

[dead]

“AI” in title gets clicks. So “AI” must be in title.

Consider ambient listening in hospitals. Encounters are recorded then AI creates a summary. Those audio recordings then can be used for any other purpose. Consider the ramifications, A panopticon of metrics derived from AI. Imagine a AI used as a review tool across all ecounters a nurse performed in a year. IA will transform how data is stored in healthcare. It's going to move towards the data lake model storing information in its raw form for post analysis.

I understand the frustration, but it's justifiable for the public to be concerned about AI in general because the novelty of the technology means that boundaries between beneficial and problematic usage is not yet stable and well defined. For existing technologies consensus on that has usually been reached, although it changes over time. In many cases we may not yet have enough information to decide.

Nor is this a trivial decision. AI has the potential to change society and economic relations as profoundly as the industrial revolution, or the invention of the printing press.

These boundaries are also contested, as interests which benefit from a particular application are different from those whose interests are harmed. Society needs to identify which usages have a net benefit.

It also needs to define which usages cause "absolute" harms which is will consider unacceptable regardless of benefits to some parties. Such as, potentially, reductions in personal autonomy, increased leverage or dominance by government or private interests.

Not only that, but data and models which were collected/ built for one purpose can easily be adapted for others.

This is also basically now happening all at once in many domains.

In short, you can expect there to be tension over these boundaries for some time. It's not realistic to expect that others will agree with your personal perception of which applications are "obviously" unproblematic.