I do not think this is fair. What is fair is at first hint of a mental distress, any LLM should completely cut-off communication. The app should have a button which links to actual help services we have.

Mental health issues are not to be debated. LLMs should be at the highest level of alert, nothing less. Full stop. End of story.

▲

freilanzer 6 days ago | parent | next [-]

So, you want an LLM to act as a psychiatrist and diagnose users whether they're allowed to use it or not?

▲

blackqueeriroh 6 days ago | parent | prev [-]

Which mental health issues are not to be debated? Just depression or suicidality? What about autism or ADHD? What about BPD? Sociopathy? What about complex PTSD? Down Syndrome? anxiety? Which ones are on the watch list and which aren’t?

	▲	sensanaty 6 days ago \| parent [-]
		(I've been diagnosed with pretty severe ADHD though I choose to be unmedicated) Ideally, all of the above? Why are we pretending these next-text-predicting chatbots are at all capable of handling any of these serious topics correctly, when all they do is basically just kiss ass and agree with everything the user says? They can barely handle trivial unimportant tasks without going on insane tangents, and we're okay having people be deluded into suicide because... Why exactly? Why on earth do we want people talking to these Silicon Valley hellish creations about their most vulnerable secrets?