Remix.run Logo
SwellJoe 4 hours ago

API pricing ends up being something like 20x more expensive for GPT 5.5 Pro than GPT 5.5 for actual work, even though the token cost is "only" 6x. On benchmarks where I've run both, I saw $1.12 mean per task with 5.5 and nearly $23 per task with 5.5 Pro, I guess it chews longer and harder on the problem.

If that's at all reflective of what it costs them to run it, I imagine they're in the same boat as Anthropic with Fable; they probably can't afford to offer it at subscription prices given current cost to operate it.

If 5.6 Sol Ultra has efficiency improvements (at one or more layers), and it allows OpenAI to offer a model that's competitive with Fable on the subscription plans, I'll guess a lot of folks will switch.

Fable is notably better than what came before. I watched it figure out stuff on its own over and over, on extremely hard problems, that I previously needed to guide a model to an understanding about, or work with them back and forth for several turns to figure it out together. Like, I've been reverse engineering a hardware device lately, and I've tried to tackle it a few times in the past with both some version of GPT and a couple of versions of Opus (most recently 4.7). In all cases, I barely made progress...would have gotten there eventually, probably, as I'm stubborn, but there were roadblocks constantly, with me and the models getting stumped and going around in circles in the end on every prior attempts.

Fable figured out other ways to find out what's happening, it dug into config files, found and extracted Boost-serialized data, compared that data to the observed behavior, built tools to compare the observed data with our emulated behavior, without being prompted. Would I have gotten there? Eventually, maybe. All prior models didn't; they mostly just tried the things I suggested and stopped at "well, that didn't work" or declared success after seeing results that matched their misunderstanding of the problem. I guess it's possible my prior attempts with other models had "loosened the lid" on the problem; we did already have a long list of documented "this didn't work" and a pile of tools for finding out if something worked. But, even so, I was impressed.

There probably will still be a "OK, let's rewrite this so it's not using lookup tables to precisely simulate the hardware behavior in software, because we don't need the noise, too" stage of the process...but, in one day with Fable, it solved a problem that I'd banged on for at least a week or too in the past with very little real progress. I don't think the models write exceedingly good code, even the best ones, but it sure does figure shit out quick.