Seems like motte and bailey fallacy. They say their models are good (the motte), therefore their models must one-shot everything to perfection (the bailey).

Besides, other people's claims about something doesn't give you license to abandon all critical thinking. Though it's evident they don't claim what you say they are.

▲

0x457 42 minutes ago | parent [-]

That's irrelevant. Questions was why people assume somthing, and answer is because that's how it advertised.

To be clear that's not what I'm thinking, even Fable 5 produces some hilariously bad results under some conditions and sonnet 5 produced great results under others.

	▲	hombre_fatal 39 minutes ago \| parent [-]
		> because that's how it advertised. But you didn't provide the evidence for that. You shared some links and then admitted they didn't claim it. It kinda seems like "because I think they're a little too positive about their product, I can set my expectations to anything I want and la-la-la it's their fault." And I don't see the problem with agents building test scaffolding as they go. It might be too defensive at times, like testing a shell script you don't run often, but big deal. It's kinda cool imo, and it's trivial to make it stop.