That benchmark methodology isn't great, but regardless, LLMs can be trained to play Chess with a 99.8% legal move rate.
That doesn't exactly sound like strong chess play.
It's enough to reliably beat amateur (e.g. maia-1900) chess engines.