| ▲ | jameshart 4 days ago |
| I think it’s a disagreement about what ‘predict’ means. The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on. Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful assistant’ or ‘sets of changes to a codebase’. And that isn’t really ‘prediction’, so much as ‘generation’. It’s not been tuned to ‘guess the next token right’. It’s been tuned to generate the token that leads to it ultimately scoring highest on its reward function. It’s not predicting the token, it’s predicting the reward. |
|
| ▲ | saghm 4 days ago | parent | next [-] |
| That seems overly pedantic to me. If I asked you "What's your prediction for the Super Bowl?", I'm pretty confident you would infer that I mean predicting the outcome, not the event itself. |
| |
| ▲ | jameshart 4 days ago | parent [-] | | But if you are an NFL coach and I ask you to decide your next action in order to maximize your odds of winning the superbowl, while yes that does involve you having some predictive ability to think about what impact your actions would have on your odds of winning the Super Bowl… I don’t think you would call the process that you use to decide that next action ‘prediction’. | | |
| ▲ | saghm 4 days ago | parent [-] | | I don't find the fact that I don't call any humans "action predictors" to be a particularly meaningful insight because my rationale is that it's a weird thing to call a human; football coaches can do plenty of other things besides just coaching football. | | |
| ▲ | jameshart 4 days ago | parent | next [-] | | Yep, we’re all just putting one foot in front of the other, hoping we’re doing the right thing to bring about the outcomes we want, trying our best. But that’s the point: so is an LLM. Putting one token in front of another, hoping it’s doing the right thing to bring about the rewards it’s trained to… trying its best. So yeah, not ‘next token predictors’. ‘Next token tryers’ maybe. | | |
| ▲ | saghm 4 days ago | parent [-] | | There's a pretty huge difference in our understanding of the methodology of how LLMs make decisions and how humans make decisions, so I don't understand why you're arguing that anything about how humans make decisions is relevant to the terminology we use for LLMs. |
| |
| ▲ | danielmarkbruce 4 days ago | parent | prev [-] | | Predict implies you don't control a situation. That's the difference. |
|
|
|
|
| ▲ | grey-area 4 days ago | parent | prev | next [-] |
| This is correct for areas where they have been intensively trained to be right, but the training covers a tiny slice of the space of text the LLM must produce and is just adjusting the weights a little. The corpus does still weigh heavily. That’s how they can reliably produce grammatically correct text. That’s also why they sometimes produce nonsense even in domains they are trained on, and more often where there was no training. For example ask it for a recipe for rock pizza or glue pizza or whatever and if it had not been specifically trained on it or had guardrails introduced, but has some nonsense in its dataset, it will reproduce the nonsense. |
|
| ▲ | leobg 3 days ago | parent | prev | next [-] |
| Or about what the word “is” means. If you’re Bill Clinton. :-D |
|
| ▲ | KETHERCORTEX 3 days ago | parent | prev | next [-] |
| > that isn’t really ‘prediction’, so much as ‘generation’ It's both. Generation in a predictive way. That's what G and P in GPT stand for. |
|
| ▲ | danielmarkbruce 4 days ago | parent | prev | next [-] |
| yes, it's exactly this. And it's not a trivial distinction. |
| |
| ▲ | grebc 4 days ago | parent [-] | | It’s deterministically trivial. | | |
| ▲ | danielmarkbruce 4 days ago | parent [-] | | Post train a model, you'll be able to determine it is not. | | |
| ▲ | grebc 4 days ago | parent [-] | | The blog post itself says one thing, but then demonstrates the exact thing they’re arguing against. If you can’t grasp that logic gap then there’s no point discussing further. | | |
| ▲ | garrinm 4 days ago | parent | next [-] | | I try to make 3 claims in the post, it was a bit clumsy I'll admit that. 1. At inference time, LLMs emit one token at a time given the prior tokens. This looks like prediction and I concede that. 2. During pre-training, LLMs predict the next token and compare to the actual next token in the training data. This is the classic setting for ML predictions. And I think its meaningful, the model really is predicting what the ground truth next token will be in the data. 3. During post-training, in the case of RLVR, there is no ground truth next token. In pretraining, the question is "what token actually came next?". In RLVR, the question is "what sequence of actions gets a high reward?" And the whole point is that thinking about the RLVR is important. A mental model that stops at 1 or 2 is incomplete and doesn't capture what drives LLM tokens. | | |
| ▲ | grebc 4 days ago | parent | next [-] | | My understanding about your third point is the LLM generates lots of different answers, then they’re ranked according to some computation the creators came up with. I’m still not sure what doesn’t qualify any of that as a prediction, and I’ll be more blunt: a guess. | | |
| ▲ | danielmarkbruce 4 days ago | parent [-] | | A guess at what though? One guesses at truths they don't know, or events that haven't happened yet. What is the model guessing? | | |
| |
| ▲ | danielmarkbruce 4 days ago | parent | prev | next [-] | | Probably the easiest way to describe an LLM that it's a policy. There is a reason that word has stuck in RL. And it's not just RLVR. RLHF has been going on for years and years. LLMs have not been "next token predictors" for probably 5-6 years. | |
| ▲ | what 3 days ago | parent | prev [-] | | It’s still just predicting the next token though just with a different reward between 2 and 3. |
| |
| ▲ | danielmarkbruce 4 days ago | parent | prev [-] | | Nope, it doesn't. No logic required, you can just build an LLM yourself, including post training. You'll see that predicting the next token isn't something the model does or is optimized for in RLHF or RLVR. You can hand wave all you like, but you have never done it. | | |
| ▲ | grebc 4 days ago | parent [-] | | Yes, no logic is necessary for LLM adherents we're all finding out. Carry on good soldier. | | |
| ▲ | danielmarkbruce 4 days ago | parent [-] | | If you haven't built one, and don't understand how they work, why comment? | | |
| ▲ | grebc 4 days ago | parent [-] | | You don't need to build a car to understand one. That you tie yourself up in knots of fancy acronyms instead of plain words and that your argument boils down to semantics of the word prediction, it's pretty clear what is up brother. | | |
| ▲ | danielmarkbruce 4 days ago | parent [-] | | Lol, sure, just read a blog post and you'll understand how a car works....It's very simple.... | | |
| ▲ | doc_ick 4 days ago | parent | next [-] | | Just like how reading a math book doesn’t teach you math, why do they make us read anyway? (Sarcasm) if reading a blog post didn’t teach someone how a car works how come it “can” work for next token predictors | | |
| ▲ | danielmarkbruce 3 days ago | parent [-] | | Mine was sarcasm. People who actually understand cars have built them. Until you build something, you don't understand it. | | |
| ▲ | grebc 3 days ago | parent [-] | | Now you’re claiming people don’t understand unless they build something. Boy, oh boy, do you keep digging your logic hole that much deeper. As mentioned earlier, Sam thanks you for your obfuscation efforts while his equity keeps going up. The swindle continues. | | |
| ▲ | danielmarkbruce 3 days ago | parent [-] | | I'm not the one hiding behind a fake name. If you want to understand how this stuff works, there are totally decent books about building them from scratch. It's not that hard, and you'll likely find it interesting. Sebastian Raschka and Nathan Lambert have good books out, and the Allen Institute has available all the code and data they have used for several projects. | | |
| ▲ | grebc 3 days ago | parent [-] | | Now a fake name accusation is thrown by someone with three first names. Keep digging that hole, I’m sure you’ll surface somewhere with some sunshine. |
|
|
|
| |
| ▲ | _superposition_ 2 days ago | parent | prev [-] | | Lol isn't this what llms do?
Did you not just undermine your entire argument? |
|
|
|
|
|
|
|
|
|
|
| ▲ | search_facility 4 days ago | parent | prev [-] |
| Well said! |
