| ▲ | AmazingTurtle 4 hours ago | |||||||
gpt-6-astra is a bitch, it constantly scope creeps itself with "yet another thing" to give it that darn polished lick. the results are eventually a little bit better but at what cost? let's do the math. gpt-5.6-sol: 1x base gpt-6-astra 2.5x base in subscription then gpt-6-astra tends to spawn subagents a lot, often with all kinds of models such as gpt-5.6, 5.3-codex etc., which is neat. it's a good coordinator but even more cost. and then it tends to run _full test suites_ over an over again (each costs like 15 minutes) just to verify that _one test_ was fixed etc., and does so for as long as until the test is fixed, eventually accumulating 2 hours or so. yesterday I assigned it a task to rebase my changs in a repo onto the latest upstream changes. while gpt-5.6-sol consistently took like an hour to do so end-to-end, astra ran for more than 6 hours and still wasn't done. it kept finding "one more thing" that was goldplating that I didn't ask for. | ||||||||
| ▲ | bob1029 3 hours ago | parent | next [-] | |||||||
> each costs like 15 minutes I've got a custom agent loop that will reuse unit testing results if no apply patch operations occurred since the last invoke. Wall clock time isn't something I would put on the AI provider. That's entirely a consequence of the system that you've brought to the party. | ||||||||
| ||||||||
| ▲ | weird-eye-issue 4 hours ago | parent | prev | next [-] | |||||||
They don't always have a great concept of time so for something like running a full test suite that takes a long time you should just tell it not to do that | ||||||||
| ▲ | 4 hours ago | parent | prev | next [-] | |||||||
| [deleted] | ||||||||
| ▲ | djmips 3 hours ago | parent | prev [-] | |||||||
have you ever worked for a big company where that's the status quo for any tiny change... hours on _full test suites_ over and over again. | ||||||||
| ||||||||