My personal view is that we humans are all too easily drawn into thinking "this would be a danger to other people, but I can handle it".

I believe that if you are in apsychological state such that the input from an LLM could pose a risk, you would also have a much reduced ability to detect and handle this, as an effect of your state.

That’s how people dig deeper and deeper holes and it becomes much harder to exit them. “I’m immune to propaganda” and then go out and buy a Disney themed shirt.

Therapy is a bit different though. It's meant to make you think. Get your mind unstuck from the loop or spiral it's in. Generally you will know what's wrong but your mind keeps dancing around it. There's a lot of elephants in the room. In that sense it doesn't quite matter that much if it tells you to do something outrageous. It's not like you're going to actually do that, it's just food for thought. And even an outrageous proposition can break the loop. You'll start thinking like oh no that's crazy. Maybe my situation isn't so bad.

The problem is when you start seeing it as an all knowing oracle. Rather than a simulated blabbermouth with too much imagination.

In general it's been very positive for me anyway. And besides I use it on myself only. I can do whatever I want. Nobody can tell me not to use it for this.

Even if it just tells you (sometimes incorrectly) that nothing is wrong and just sides with you like a friend, even that is good because it takes the pressure of the situation so reality can kick in. That doesn't work when stress is dialed up to the maximum.

It also helps to be the one tuning the AI and prompt too. This always keeps your mind in that "evaluation mode" questioning its responses and trying to improve them.

But like I said before, to me it's just an augmentation to a real therapist.