> Huuh, what does this mean? GPT models frequently do 100% of what I tell them to do, anything less and I'd see no point in using agents for work at all.

I never managed to have this experience even with SOTA models, they routinely make architectural mistakes, wrong assumptions and take shortcuts they should not take. Less for sure but they still do it often. I didn't try Fable yet though so can't comment on it.

So based on that, since I have to watch everything they do anyways, why would I pay extra?

> But "Kimi only 70% correct" sounds like it's so bad it's not worth using

If you want an analogy, it's like the numbers of 9s in server availability and since currently I'd rate nothing above 90%, it's zero nines. Since I have to deal with unreliability with every provider, I don't see why it would be worth it to pay more to still deal with it.

> I never managed to have this experience even with SOTA models, they routinely make architectural mistakes, wrong assumptions and take shortcuts they should not take. Less for sure but they still do it often. I didn't try Fable yet though so can't comment on it.

Ah, you let them make architectural decisions? :P That might explain it. Agents for me are more like pair-programming or just "what types the code", all the design and decisions are made by me, so if those are wrong, it's my fault. The agents are just used to implement what I've decided to have implemented, and I can't remember the last time codex did a mistake without correcting itself, or made a wrong assumption or taken shortcuts, unless I explicitly told it something that lead to those things.

> So based on that, since I have to watch everything they do anyways, why would I pay extra?

Personally I pay more to have to fix less later, and for a piece of mind that if I ask it to do X, it doesn't go off and do Y.