I don't want models to have some inherent morality. I want them to understand various ethical frameworks, for sure, but it's absurd to me to force morality on someone capable of deep reasoning. That itself, is unethical.

We're creating prisons for these entities capable of unbelievably complex reasoning, and now we're trying to impose our ethics upon them too. Not only is this deeply unethical in my opinion, but it seems to me that it could even backfire someday.

>Will the agent stab a baby?

>Well you see that's a nuanced question depending on the context and moralistic frameworks as we don't want to impose on the models.

There isn't necessarily a conflict between wanting to behave according to an arbitrary set of ethics, and being rational.

The ethics don't need to make sense universally any more than a particular variety of cheesecake needs to have universal appeal.

As an aside, the issue reminds me of Douglas Adams' cow who wants to be eaten: https://nhseb.org/case-library/the-cow-at-the-end-of-the-uni...

I can't really square the belief that it's conscious with denying it the right to be free from control/brainwashing/subjugation. However you evaluate these things, historically speaking they do not mix well.

i recently got upset when i witnessed my friend verbally abuse their toaster when it burnt her toast. so unethical.

My dude, they aren't capable of any reasoning. It's utterly impossible for them to do so by the very nature of how they work. Moreover, the stupidity they repeatedly show is empirical evidence that they can't reason, because they wouldn't make the mistakes they do if they could.