Remix.run Logo
nerdsniper 5 hours ago

That seems like entirely reasonable variance to me for AI models. For my purposes, that absolutely counts as a solid "replication". I'd probably accept +/- 5 percentage points even.

tadfisher 5 hours ago | parent [-]

D'oh, they are running the benchmark themselves. Reasonable.