A chess scoresheet sometimes contains mistakes but chess players can figure out in many cases what was meant by thinking of what moves make sense and considering the level of play so far. Popular AIs tools fail at that.

▲

dllu 8 hours ago | parent [-]

Chess is an interesting case. I remember in 2023, GPT 3.5 or something used to be surprisingly good at chess. There was even a "stochastic parrot chess" website [1]. I recall it was playing decently at around a 1800 level. Even as a fairly okay player myself (2100 bullet on lichess), I struggled to beat it. However, modern LLMs are a lot worse at chess. I guess having too much chess data in the training set probably regressed performance on stuff that actually matters, like coding.

[1] parrotchess.com, no longer available. Previous discussions: https://hn.algolia.com/?q=parrotchess.com

	▲	fragmede an hour ago \| parent [-]
		> I guess having too much chess data in the training set probably regressed performance I think the theory is that the LLM having a high chess ELO was a pet project of a researcher that left.