We have existing precedent that encouraging someone to kill themselves can result in you being criminally responsible. Is software doing its best to be human that different?

https://en.wikipedia.org/wiki/Death_of_Conrad_Roy