If you employ a few hundred nurses, how exactly would you evaluate how well they show empathy?

You can't rely on asking the customer. When they're upset (they often are in these calls), they'll lean towards the negative regardless.

I don't know how well these AIs evaluate, but if they're even a little bit good, it makes sense to use it to screen for outliers, then have a human listen to those outliers and judge.

"You can't ask people because the experience is so universally terrible they'll just tell you it's terrible" isn't really an argument against surveys, it just means you need more specific questions they'll be fired up to answer

> how exactly would you evaluate how well they show empathy?

How would you want yours rated? By someone you have communicated with, or some data centre somewhere?

Actually, you can rely on the customers. They're the only ones that can tell you.

I am quite heavily in AI, and I would say I am pro AI. However this use-case for AI is putting AI in the wrong position. AI should be in service to all humans. An administrator building out a middle management KPI based on AI is a misapplication of AI.

Hospital systems are incentivized to avoid the real problems with healthcare. People want timeliness and they want quality care which hospital systems are not incentivized towards in the US. The incentives are profit, which given budgets means corners cut.

Triaging is an opaque system to the patient. It's an important process to doll out finite resources but it also very frustrating to be told, "soon" when you've been waiting 15 hours to see someone. Frankly, if I were King for a day, the first thing I would do is break up the monolithic hospital systems and build out more urgent care.

I would also try to find a way to facilitate transferring less critical patients from ERs to urgent care centers. Right now a hospital won't take the risk, especially if you are sitting in waiting room because beds are full. You can't easily punt a patient because them leaving would be against medical advice.

> You can't rely on asking the customer

It’s not even been five years of AI, and we’ve already arrived at the point where the human is wrong and the AI is right.

Mind you this is in an area where the benchmark is the opinion of the human ! So if the customer is saying you’ve shown enough empathy but AI says you haven’t, then you take opinion of the AI?

Soon we’re going to have a situation where the patient is breathing, but the AI says he’s dead.

Not asking the customer because you're afraid they'll tell you they're upset is a good indicator that you should do it more, and fix the issues.

You can ask the customer enough times that unreasonable customers or surveys are averaged out.

A good question might be "why are you upset?"

There's a great academic book called "Stupid Rules". Excessive rules (or KPIs - often rules in disguise) exist because we don't like authority.

Get the doctor to assess the nurse. Or the head nurses if you don't trust doctors. The nurses have managers, and if none of the doctors or head nurses can be trusted with a simple matter like assessing whether nurses are doing their jobs then you got bigger issues.

Oh no, the boss might play favourites if it's not an objective measure! Oh the injustice /s

But stupid rules or KPI also allow favourites. You can use an officious 30 point checklist and play favourites while ticking boxes. You can even rig "objective" data by controlling other factors (e.g. giving someone difficult customers do deal with).

Yeah, data driven would be nice, if you have good data. But data driven is a power tool. You don't measure SLOC or reward token use in software because of perverse incentives.