i'm not happy with how openai is trying to pit 5.6 sol as a cheaper equivalent to fable here

for one thing, they said that on AA, sol is "within one point of fable" at 58.9 vs 59.9 but don't clarify that the latter is with safeguards where ~8% of the tasks got routed to opus

i'm not rooting for either and genuinely think that the token efficiency and cheaper price are important but this sort of thing just feels disingenuous :-/

This is especially interesting because IIRC the AA benchmark is calibrated so that 1 point and greater difference is statistically significant.