GRPO increases the likelihood of samples that are better than average, not just the single best, and decreases that of samples that are worse than average. This method doesn't even involve an explicit likelihood, so it's a completely different mechanism.

A comparison with minibatch optimal transport is in appendix A.2 of the paper.

▲

zimablue22 3 hours ago | parent [-]

You're right about the original GRPO proposal, but there are simplified variants that do just use best of K sampling.

GRPO (or GRPO like approaches) for diffusion/flow matching similarly can be likelihood free.

▲

yorwba 2 hours ago | parent [-]

Without anchoring on the group average it's hardly Group-Relative and without likelihood there's no Policy; that doesn't leave much of the original GRPO unchanged...

Do you have a particular paper in mind that does likelihood-free best-of-K but just calls it GRPO?

▲

zimablue22 2 hours ago | parent [-]

Any of the Best of N papers that exploded in popularity after GRPO.

Likelihood is not fundamental to the spirit of GRPO, any exploratory mechanism would work.

That sequential LLMs have a step-wise probability is convenient but not critical to this approach (where rejection sampling is widely used in diffusion models).

	▲	kamranjon an hour ago \| parent [-]
		Do you have an example? Would love to read one.