| ▲ | imenani 3 hours ago | |
The relation to current RLVR methods I think is interesting, they do discuss it a bit but I would be curious to see more about this as well. Quote from the paper: Exploration beyond Pretraining. The mode collapse XMs address during pretraining also often shows up in post-training, where RL fine-tuning is known to sharpen models onto a narrow set of behaviors [76]. Recent fixes such as pass@krewards [77] and best-of-N-aware fine-tuning [78] can be seen through our lens as Forward XM, with a verifier standing in for ground truth data. These fixes act only during post-training, though; pretraining with exploration may yield base models that capture more modes in the first place, leaving RL more to select among. | ||
| ▲ | zimablue22 3 hours ago | parent [-] | |
That's fair. On the other hand, Minibatch OT (optimal transport) was one of the more fundamental advancements early on in flow matching & rectified flow models. This best of K approach effectively discards matches that would otherwise be preserved by optimal transport assignment. I still think the authors here are too broad regarding their claimed novelty. | ||