There is the art of saying no: https://dl.acm.org/doi/10.5555/3737916.3739489

It is possible to create (subjective) reasoning traces like https://huggingface.co/datasets/Bachstelze/ethical_coconot_6...

And train or adapt a model to it: https://huggingface.co/Bachstelze/olmo-7b-ethical-reasoning-...

This is just a little proof of concept, though it is maybe the direction you are looking for?!