| ▲ | YeGoblynQueenne 2 days ago | |
You're right of course and my bad but in my defense even Sutton & Barto lapse into using the term interchangeably (I caught them at it while reading the book earlier). >> And, the expected return from a state is often estimated rather than an explicit trajectory run out all the way. A value function can estimate expected return without explicitly predicting the future states or individual rewards that make up that return. I think rather that's the job of the value function, to inform the policy of the, well, value of a state, or state-action pair. Somehow the policy needs to know what a good action is to take in the current state, or what is the best action to take. And of course it doesn't have to run the entire episode before it does that: but that's what I mean that it predicts the next move and not just the outcome of the entire game. | ||
| ▲ | danielmarkbruce 2 days ago | parent [-] | |
But the next move is self evident if you have a prediction of the value of being in each of the states possible. | ||