Triage exists. And when the shit hits the fan, it takes the form of "this one isn't worth trying to save, but this one might be".

It's callous and cold, and it results in more lives saved than trying to save everyone.

Does ChatGPT, on the net, save more people than it dooms? Who knows. Plenty of anecdotes both ways, but we wouldn't have the reliable statistics for a long time.

Triage exists because in a mass casualty event, medical resources need to go to the most urgent cases first.

Triage is not a punch card that says if you drag 9 people out of a burning building, you can shoot someone on the street for free.

Sure, triage done by a human exist and they are generally absolved from taking that decision that could kill someone (to save others). Because the situation calls for it and would otherwise result in more deaths. But other than the situation calling for it, healthcare professionals are held to a certain standard and can be litigated against for medical malpractice.

That is not the case for ChatGPT apparently, and OpenAI should be held responsible for what their models do. They are very much aware of this because they did fine-tune GPT5 to avoid giving medical advice, even though it's still possible to work around.