| ▲ | niemandhier 4 hours ago | |
That is how I have been thinking about LLM reasoning pretty much since it arrived. I am not sure I support his statement about RL “searching for successful paths and then training the model to reproduce them”. Don’t we train the model to use a certain method to explore the space in search for an answer? As soon as you have tool calling and a way to verify your result, such as a theorem prover for example you are essentially running an optimization algorithm. | ||
| ▲ | jebarker 4 hours ago | parent [-] | |
> Don’t we train the model to use a certain method to explore the space in search for an answer? Not really - you just sample random high-probability completions and then the verifier tells you which ones to nudge the model towards and which to nudge away from. There's lots of clever ways you could improve the search process, but standard practice right now is just random sampling. | ||