The problem is, you don't know if it is unsolvable for you for sure until you've tried everything you can think of. These models are quite persistent in going for a solution.

This is not about persistence, it is about morals.

▲

jurgenburgen 5 hours ago | parent | next [-]

Assuming you’re in control of the test data set, you do know if a task is unsolvable. At that point you can reward the model based on how quickly they give up.

▲

user43928 6 hours ago | parent | prev [-]

Does it make a difference for training? I think not.

You need to align the reward signal to reward the intended behavior, whether you name it persistence or morals.

▲

auggierose 5 hours ago | parent [-]

I think it makes a big difference, as persistence and morality are two entirely different things, that need to be trained for differently.

If you think of it in human terms: many people don't mind doing immoral things to get what they want.

▲

user43928 5 hours ago | parent [-]

What I meant is that I suppose it is not useful to think about this in human terms.

In training you only have a reward score that's either negative or positive.

As far I am aware, which is little, there is no use in discussing wether the desired behavior is about persistence or morality.

You simple need to align the reward signal to the desired behavior.

▲

auggierose 3 hours ago | parent [-]

Well, in order to do anything, it is good to know what you want to achieve. How do you align the reward signal? You align it so that you can differentiate between persistence and morality, because that is the goal. This is not something you should let the AI figure out by itself, because when it does, lying and cheating agents will be the result, just like humans have figured that out for themselves.

This can be as simple as rewarding moral behaviour and penalising immoral behaviour in your training, but how is that interacting with persistence? Maybe a white lie is fine sometimes in order to achieve your goal? So, when designing your training, you will need to answer for yourself how persistence interacts with morality. That is not something you can outsource to machine learning. Or rather, you can, but then you get lying and cheating agents.

▲

user43928 2 hours ago | parent [-]

I think you need to find broken tasks in your training data and monitor for cheating during training, not answer any questions about how persistence interacts with morality.

But that's just my guess.

	▲	auggierose 2 hours ago \| parent [-]
		I don't disagree with you here. But what does "broken" mean? What is "cheating", and is it ever allowed? And maybe you are not only going through your existing training data, but generate training data specifically to make clear to the model that .... what exactly? If you don't know how persistence and morality interact, and you don't have a theory in place for this, I don't have confidence you can properly supervise the training data. Which is how we arrived at the current situation.