ChatGPT medical advice brought man 'to brink of death', lawsuit alleges

sheepscreek 15 hours ago | parent | next [-]

I really hope the lawsuit does not end in OpenAI tightening the “safety” and “security” of their models into oblivion. I hope they understand that a lobotomized model is as good as my Casio watch.

I want my models to give it to me straight without patronizing me. I’m an adult and I take responsibility for my own actions.

▲

lardosaurusrex 14 hours ago | parent [-]

That's the issue here tho.

You do not own the model; they do. Ergo it is their advice and so they can be held accountable fo it.

▲

fhn 13 hours ago | parent [-]

so if you give your friend medical advice and they are injured, can they sue you?

	▲	lardosaurusrex 5 hours ago \| parent \| next [-]
		Your friend is not a multi-billion dollar gigacorp what in the goddamn hell are you babbling about? Do you think that arguement would work on a judge? Good lord. Thanks for announcing to everyone that your AI psychosis is so bad that you consider LLMs to be your friend(s).
	▲	dnemmers 9 hours ago \| parent \| prev [-]
		I'm sure they could try.

▲

arm32 15 hours ago | parent | prev | next [-]

Why are people even allowed to dress up ChatGPT... an autoregressive transformer... as a faith-based holistic doctor? Anybody could see this going wrong from a hundred miles away.

	▲	throw310822 14 hours ago \| parent [-]
		A few days ago a family member insisted to be brought to the ER because Claude told him. Result: he was immediately hospitalised and received surgery the following day to remove a tumour that was compressing his spine. Claude was very clear that he had to act as quickly as possible to lower the chances of permanent damage.

▲

harvey9 15 hours ago | parent | prev | next [-]

All the chat bots have a very discreet disclaimer at the bottom of the page. I want to say it should be bigger text and more strongly worded but like most people I would just mentally tune it out regardless.

	▲	Arnt 15 hours ago \| parent [-]
		The LLM I use is fairly quick to retreat into extra disclaimers. "Not sure about this, this is from memory and you ought to check", etc. Don't they all do that? Is it just because of my prompt (which does say to push back if I appear to have bad ideas)? It seems possible that people tune out disclaimers and caveats in the middle of the answer, too.

▲

ButlerianJihad 3 hours ago | parent | prev | next [-]

Ironically, the given name "Claude" means "crippled" or "lame". Students of the Roman Empire may recall the disabled Emperor Claudius.

▲

sharts 8 hours ago | parent | prev | next [-]

There are no bounds to human stupidity

▲

heaney-555 15 hours ago | parent | prev | next [-]

This was with GPT-4o (ancient now by AI standards). The reason OpenAI refuses to bring it back, despite all the demand, is that when the conversation gets long enough it starts doing weird things like this.

Subsequent models have almost entirely eliminated this kind of thing.

	▲	siva7 14 hours ago \| parent [-]
		Even GPT-5.4 is ancient compared to intelligence of 5.6 although those are only 4 months apart. We're talking about at least twice the capabilities and that's scary to achieve in such a short timeframe.

▲

lardosaurusrex 16 hours ago | parent | prev | next [-]

The amount of times these LLMs randomly refuse to answer basic questions because "omg safety" only to turn around and do this f￭￭￭ing nonsense is obscene.

I'm not blaming the LLM because it's just a pile of math glued together with hubris and anxiety but why is its default response not "I am not a medical professional and you should see a doctor right away if you have any concerns"?

	▲	fhn 13 hours ago \| parent [-]
		because they get sue otherwise?

▲

fhn 13 hours ago | parent | prev [-]

this is number one bullshit. You ask a medical professional for medical advice. Getting medical advice from AI or anywhere else is your own fault.