Remix.run Logo
scrlk 5 hours ago

Artificial Analysis is reporting that 6 Luna scores 2 points lower on their coding index than 5.6 Luna, but is 60% cheaper:

> In the Coding Agent Index, Sol improves but Luna regresses: In OpenAI's Codex harness, GPT-6 Sol (max) scores 57 in the Artificial Analysis Coding Agent Index, up 2 points from GPT-5.6 Sol (max), with gains in Terminal-Bench 4.0 (43% vs 37%) and SWE-Atlas-QnA (58% vs 54%). At $2.99 per task it costs ~50% less than GPT-5.6 Sol (max) and sits on the Pareto frontier of Coding Agent Index vs Cost per Task. GPT-6 Luna (max) scores 41, down 2 points from GPT-5.6 Luna (max), with lower scores in SWE-Atlas-QnA (44% vs 49%) and DeepSWE v1.1 (64% vs 66%), at ~60% lower cost per task.

https://x.com/ArtificialAnlys/status/2102462962758033624

Given that they had to discontinue sales of the 20x Pro plan after the Astra release due to compute constraints, I wonder if 6 Sol & Luna are smaller vs their 5.6 counterparts?

scrollop 4 hours ago | parent | next [-]

Can we trust AA anymore after the last debacle a week or two ago?

6thbit 2 hours ago | parent [-]

wait what debacle?

Readerium 5 hours ago | parent | prev [-]

Yup more like a 5.7 than a 6

5 hours ago | parent [-]
[deleted]