Classifiers and LLMs get very different training and objectives, it's a mistake to draw inference from MNIST for coding agents or LLMs more generally.

Even within coding, their capability varies widely between context and even runs with the same context. They are not better at judgement in coding for all cases, def not

A lot of the context is not even explicit, unlike the case for toy problems like MNIST.