I think they will all be minor going forward, feels like the major improvements have all been made and we'll only see incremental improvements from here on out. Maybe I'm wrong but we'll see.

▲

spelk 13 days ago | parent | next [-]

Hard to say. People made the same prediction a year ago because we supposedly ran out of training data. There could be indefinite rapid compounding improvements so long as there's free money out there.

	▲	jmalicki 13 days ago \| parent [-]
		With RLHF and RLVR we are creating tons of new training data, that is much more focused than reading the Internet. Annotation shops are doing many billions per year in revenue creating newer data, and a lot of it is highly complex, focused on rewarding multi turn agentic trajectories.

▲

Eufrat 13 days ago | parent | prev | next [-]

I think one of the challenges is that the models were all initially trained on the entire Internet (or as much as they could gather) and now they’re having to deal with an increasing amount of the Internet being AI generated content which may be why GPT-5.5 started being obsessed with goblins and you start seeing amusing things in the system prompt trying to get the model to stop bringing them up.

▲

conradkay 13 days ago | parent | prev | next [-]

I think there's just less time between model releases now

▲

chandureddyvari 13 days ago | parent | prev [-]

Wasn't Mythos a step change improvement?