They are also weirdly bad at Brainfuck which is basically just a subset of C.

culi an hour ago | parent | next [-]

Yeah well they also still struggle with "4 + 6 / 9" so I'm not sure why anyone is surprised with these findings

astrange 6 hours ago | parent | prev [-]

BF involves a lot of repeated symbols, which is hard for tokenized models. Same problem as r's in strawberry.

Interesting. So why do the models seem to handle deeply nested Lisp expressions just fine?

	▲	kgeist 5 hours ago \| parent [-]
		Probably because there's a ton of code that deals with nested parentheses across languages in the training data, and models have learned how to work around tokenization limitations, when it comes to parentheses.