| ▲ | SilverElfin 12 hours ago | |||||||
How did you get all this info about how each is trained? Is that something they admit now or is it through leaks? | ||||||||
| ▲ | jychang 11 hours ago | parent [-] | |||||||
Deepseek? It's literally in their research papers. OpenAI? The OpenAI head of research @markchen90 straight up admitted it in a podcast. https://x.com/petergostev/status/1995744289079656834 "In the last 2 years we've put so much resourcing into, into reasoning and one byproduct of that is you lose a little bit of muscle on pre training and post training." "In the last six months, @merettm and I have done a lot of work to build that muscle back up." "With all the focus on RL, there's an alpha for us because we think there's so much room left in pre training." "As a result of these efforts, we've been training much stronger models. And that also gives us a lot of confidence carrying into Gemini 3 and other releases coming this end of the year." Note, "alpha" in the quote above is referring to https://en.wikipedia.org/wiki/Alpha_(finance) But it's pretty clear that the last full pretrain run they've released is for gpt-4o 2 years ago*, and since then they've just been iterating RL for their models. You don't need any insider information to notice that, it's pretty obvious. *Excluding GPT-4.5 of course, but even OpenAI probably wants us to forget about that. | ||||||||
| ||||||||