| ▲ | nerdsniper 5 hours ago | |
That seems like entirely reasonable variance to me for AI models. For my purposes, that absolutely counts as a solid "replication". I'd probably accept +/- 5 percentage points even. | ||
| ▲ | tadfisher 5 hours ago | parent [-] | |
D'oh, they are running the benchmark themselves. Reasonable. | ||