| |
| ▲ | gwerbin 3 days ago | parent [-] | | Sure, you're right. But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence. | | |
| ▲ | danielmarkbruce 3 days ago | parent [-] | | The policy is not learned token by token during RLHF and RLVR. The reward model doesn't score token by token. | | |
| ▲ | gwerbin 2 days ago | parent [-] | | So? tokens are emitted one at a time according to the output distribution & sampler algorithm, and the next token distribution is a function of the preceding token sequence only. The process by which the output distribution is shaped doesn't change the core mental model, and doesn't reduce its value. It's a prediction in the jargonic sense that an inference about future values of a time series is broadly called "prediction", and it's relevant for reasoning about LLMs because they are fundamentally limited to converting tokens sequences into next-token predictive distributions, and that bears on how they can do what they do and what their limitations are. Nothing about the training process changes that. |
|
|
|
