Remix.run Logo
▲ qphe95 3 hours ago

I wouldn't say perplexity is great metric because GPT-4 and o3 have similar perplexity on this chart but o3 was a massive step improvement. Pachocki has an incentive to push perplexity as one of the creators of the original LLM paradigm but I would say we have kind of moved on from that.

▲alextheparrot 3 hours ago | parent | next [-]

Almost verbatim addressed in the post under “Perplexity as a shared metric has challenges”

▲namr2000 3 hours ago | parent | prev | next [-]

This metric is purely scoring the ""intelligence"" of the base model, not including RLVR/chain of thought capabilities. It's widely believed that the pretrain determines the ceiling that RLVR can take you to.

▲jrflo 3 hours ago | parent | prev [-]

I think it can still be a useful metric, but probably isn't the "one true" metric. The trend is pretty clear, and the acceleration of capabilities since December 2025 definitely aligns with my subjective experience.

There are probably some other axes where o3 is clearly better than o1, but with how complex intelligence is no metric will be perfect.