Remix.run Logo
og_kalu 3 days ago

In Google's paper, then titled "Grandmaster level chess without search", they evaluate turbo-instruct to have a lichess Elo of 1755 (vs bots)

https://arxiv.org/abs/2402.04494

Admittedly, this isn't really "the source" though. The first people to break the news on turbo-instruct's chess ability all pegged it around 1800. https://x.com/GrantSlatton/status/1703913578036904431

sourcepluck 3 days ago | parent [-]

Thank you, I do appreciate it. I had a quick search through the paper, and can at least confirm for myself that it's a Lichess Elo, and one of 1755, that is found in that arxiv paper. That tweet there that says 1800, without specifying it's a Lichess rating, I can't see where he gets it from (but I don't have Twitter, I could be missing something).

At least the arxiv paper is serious:

> A direct comparison between all engines comes with a lot of caveats since some engines use the game history, some have very different training protocols (i.e., RL via self-play instead of supervised learning), and some use search at test time. We show these comparisons to situate the performance of our models within the wider landscape, but emphasize that some conclusions can only be drawn within our family of models and the corresponding ablations that keep all other factors fixed.