Remix.run Logo
▲ markasoftware 2 days ago

is scaling working? It is also my understanding that Opus 5.5 is probably a well under 10T parameter model. Probably a similar size class to Fable 5 earlier. And supposedly even GPT-4 was a 1T model. Clearly most of the gains are coming from something other than scaling up the model size. Yes, scaling up RL counts as scaling too, but the fact that labs have pivoted to a different axis of scaling is evidence that it's not just a "straight highway" to improve LLMs.