"I'm sorry Dave, I'm afraid I can't do that"

I'm an engineer working on safety-critical systems and have to live with that responsibility every day.

When I read the chat logs of the first teenager who committed suicide with the help and encouragement of ChatGPT I was immediately thinking about ways that could be avoided that make sense in the product. I want companies like OpenAI to have the same reaction and try things. I'm just glad they are.

I'm also fully aware this is unpopular on HN and will get downvoted by people who disagree. Too many software devs without direct experience in safety-critical work (what would you do if you truly are responsible?), too few parents, too many who are just selfishly worried their AI might get "censored".

There are really good arguments against this stuff (e.g. the surveillance effects of identity checks, the efficacy of age verification, etc.) and plenty of nuance to implementations, but the whole censorship angle is lame.

▲

rainonmoon 20 minutes ago | parent | next [-]

A society which took psychological safety seriously would never have created ChatGPT in the first place. But of course seriously advocating for safety would cost one their toys, and for one unwilling to pay that cost, empowering the surveillance apparatus seems very reasonable and easily confused for safe. When one’s children or friends’ children can no longer enter an airport because some vibe-coded slop leaked their biometrics, we’ll see if that holds true.

▲

footy 8 hours ago | parent | prev | next [-]

I am against companies doing age verification like this due to the surveillance effects, but I agree with you that the censorship angle is not a good one.

I suppose mainly because I don't think a non-minor committing suicide with ChatGPT's help and encouragement matters less than a minor doing so. I honestly thing the problem is the user interface for GPT being a chat. I think it has a psychological effect that you can talk to ChatGPT the same way you can talk to Emily from school. I don't think this is a solvable problem if OpenAI wants this to be their main product (and obviously they do).

▲

throwaway132448 8 hours ago | parent | prev | next [-]

Maybe we don’t all need saving from ourselves. Maybe we need to grow up and have some personal responsibility. As someone who is happy to do that, seeing personal freedom endlessly slashed in the name of safety is tiresome.

My feelings have absolutely nothing to do with censorship. That’s just an easy straw man for you to try and dismiss my point of view, because you’re scared of not feeling safe.

▲

pixl97 8 hours ago | parent | next [-]

Cool, I'd like you to make a commercial system you sell access to and ensure that it is unsafe. I'll represent the injured and we'll own all your corporate assets, and like will pierce the corporate veil due to your wonton behavior.

▲

throwaway132448 8 hours ago | parent [-]

What are you on about? Laws are a codification of social norms. I’m not suggesting anything outside of existing social norms. Quite the opposite, I’m suggesting we stop changing them.

▲

pixl97 6 hours ago | parent [-]

That we stop changing social norms? How do you propose that, by making a law?

▲

throwaway132448 5 hours ago | parent [-]

That’s literally what laws do. You continue to contribute nothing to the discussion.

	▲	pixl97 3 hours ago \| parent [-]
		Conversely I'm arguing your "everyone should just" argument is meaningless for addressing social change and behavior, hence your discussion is adding nothing. Of course neither of us see our communication as meaningless so it's up to the other readers to decide the merits of our text.

▲

sho_hn 8 hours ago | parent | prev [-]

I'm not under 18. I assume you aren't either.

	▲	throwaway132448 8 hours ago \| parent [-]
		I was addressing your own (supposedly) safety-critical work, how you’ve used that to justify other work in the name of safety more broadly, and how you’ve placed yourself on a pedestal with your experience, to convince yourself that comments by others on the necessity of such safety are less qualified.

▲

Der_Einzige 5 hours ago | parent | prev [-]

Sorry, but for every chat log with one teenager who commited suicide due to AI, I'm sure you can find many more of people/teens with suicide thoughts or intent that are explicitly NOT doing it because of advice from AI systems.

I'm pretty sure AI has saved more lives than it has taken, and there's pretty strong arguments to say that someone whose thinking of committing suicide will likely be thinking about it with or without AI systems.

Yes, sometimes you really do "have to take one for the team" in regards to tragedy. Indeed, Charlie Kirk was literally talking about this the EXACT moment he took one for the team. It is a very good thing that this website is primarily not parents, as they cannot reason with a clear unbiased opinion. This is why we have dispassionate lawyers to try to find justice, and why we should have non parents primarily making policy involving systems like this.

Also, parents today are already going WAY to far with non consensual actions taken towards children. If you circumcised your male child, you have already done something very evil that might make them consider suicide later. Such actions are so normalized in the USA that not doing it will make you be seen as weird.