In what world is 55.7 the same number as 54.8?

What variance is acceptable to publish without a retraction?

That seems like entirely reasonable variance to me for AI models. For my purposes, that absolutely counts as a solid "replication". I'd probably accept +/- 5 percentage points even.

There is randomness in LLMs. Both papers authors probably ran the bench 1-N times. Depending on that, they might select an average, max, least, etc. They might also have discarded outliers.

Like the other person said 5% variation is probably expected

I don't know who downvoted the parent or why, but it's a fair question IMHO.

The answer is there can be dramatic difference running a benchmark one time, because LLMs are not deterministic. A proper methodology would ask each question 20 times and calculate the mean correctness across experiments.

The reason is that the temperature parameter introduces random behavior.