IANAL, but:

One first amendment test for many decades has been "Imminent lawless action."

Suicide (or attempted suicide) is a crime in some, but not all states, so it would seem that in any state in which that is a crime, directly inciting someone to do it would not be protected speech.

For the states in which suicide is legal it seems like a much tougher case; making encouraging someone to take a non-criminal action itself a crime would raise a lot of disturbing issues w.r.t. liberty.

This is distinct from e.g. espousing the opinion that "suicide is good, we should have more of that." Which is almost certainly protected speech (just as any odious white-nationalist propaganda is protected).

Depending on the context, suggesting that a specific person is terrible and should kill themselves might be unprotected "fighting words" if you are doing it as an insult rather than a serious suggestion (though the bar for that is rather high; the Westboro Baptist Church was never found to have violated that).

I think the "encouraging someone to take a non-criminal action" angle is weakened in cases like this: the person is obviously mentally ill and not able to make good decisions. "Obvious" is important, it has to be clear to an average adult that the other person is either ill or skillfully feigning illness. Since any rational adult knows the danger of encouraging suicidal ideation in a suicidal person, manslaughter is quite plausible in certain cases. Again: if this ChatGPT transcript was a human adult DMing someone they knew to be a child, I would want that adult arrested for murder, and let their defense argue it was merely voluntary manslaughter.

> Which is almost certainly protected speech (just as any odious white-nationalist propaganda is protected).

Fun fact, much of the existing framework on the boundaries of free speech come from Brandenburg v. Ohio. You probably won't be surprised to learn that Brandenburg was the leader of a local Klan chapter.