| ▲ | garrinm 4 days ago | |
In the article I made 3 claims, and I agree it was a bit clumsy. 1st I say that "working forwards" in the sense of outputting one token at a time could be some form of prediction, I don't argue against that. This is what LLMs do at inference time. 2nd I say that to me what really constitutes a prediction is the pre-training. Here it's the classic setting for the word prediction in ML. The model outputs a prediction of the ground truth label: the next token. 3rd I argue that in RL there is no ground truth next token, so prediction doesn't apply here anymore. Back to your question then: you're asking points 3 and 1 are different. Working backwards from a set of win states is basically what RL does in training. Working forward from the current state is what inference does. To me there is a distinction worth thinking about. First between the mechanism at inference time and at train time. Then between what happens in pre-training vs. RL post training. | ||
| ▲ | vrighter 2 days ago | parent [-] | |
Neither is there a ground truth in rock paper scissors (the pattern varies by the player). But one would still be trying to detect patterns and predict the next "token" the opponent is going to output. | ||