Remix.run Logo
▲ zurfer 3 hours ago

The chart pretty much explains why I was freaking out about AI at GPT-4, because the trend of improvement has been so steep. And then followed a strong slow down, but now in 2026 things have really accelerated again. My hunch is really that coding is a freaking important input for AI and we might have actually entered some form of recursive self improvement. I know some people hate the term but I feel it's somewhat fine as coding is saturated (models can get better at it) and it has many levers for model training and inference (from algorithms to data).

Anyway, I think it's good idea. Would love to see all models on this chart, but honestly we might not be able to trust it going forward as it becomes a target.

▲rhdunn 2 hours ago | parent [-]

Domains like code are easier to create autonomous RL loops for as you can easily check what the LLM generates by running it through a compiler, etc..

Domains like creative writing are harder as the automated grading suitable for autonomous RL is difficult (aside from spell checking and grammar checking).