| ▲ | kalcode 3 hours ago | |||||||
Oh oh! The models actually know what happen as well if you want to dig deeper. Pre-scale brute force training hit there limits. They broke through late 2025 due to RL training techniques that taught them to fail, fail, fail to succeed instead of spin off into a corner with nonsense. Basically we started tuning them to be smarter and able to handle 2 to 4 hour tasks. I believe this will hit it's limits as well. When IDK. | ||||||||
| ▲ | SwellJoe 2 hours ago | parent | next [-] | |||||||
Given the current models from Anthropic and especially OpenAI have a hard time constructing coherent English sentences that a human can understand even while being better at code and math, I'd say they're reaching that limit; can't use a model that you can't communicate with. The Deepmind folks seem to have figured something out on that front with Neon, though, as it's both smart/capable and a clear communicator. | ||||||||
| ||||||||
| ▲ | ACCount39 2 hours ago | parent | prev | next [-] | |||||||
This graph specifically tries to measure pre-train perplexity. It measures the "foundation" of a given base model - before RL is applied to it. On long term tasks, "CoT + RL" of o1/o3 beats GPT-4.5 resoundingly. But on raw base model perplexity, GPT-4.5 is a significantly better model than any of the o-series. The curve following GPT-4.5 and o-series suggests that newer models take advantage of BOTH better base model capabilities (perplexity on held-out data) and better RL. | ||||||||
| ||||||||
| ▲ | famouswaffles 2 hours ago | parent | prev [-] | |||||||
They are reporting perpexlity on pre-trains not RL-tuned models. | ||||||||