Which model/agent/harness tool did you use? I've found what you describe was my exact experience some ~6-8 months ago, but since about a month or so the game has completely changed. Using 5.6 Sol with highest reasoning setting in Codex or Fable in Code, the models come up with a list of possible improvements from static analysis (ranked by complexity/benefit), write and run their own custom profilers and deliver significant performance improvements with barely any input needed from my side. So this is no longer a model issue, it's a user toolchain issue.

▲

qsort 9 hours ago | parent [-]

Very hard to say anything definitive on this because it's a moving target, but last time I tried models still had a distinct sense of "consistently good, sometimes great at micro, bad at macro". Similar to how, even for relatively pedestrian CRUD, they'll do code that's objectively fine at the function/file/class level but can still make a mess if you don't supervise them at least at a high-level.

▲

bonoboTP 3 hours ago | parent | next [-]

It is moving so fast that your experience with older models is irrelevant today. Sorry.

Claude Code, Fable 5, xhigh reasoning, allow it to run the full CI, end to end and benchmark, it will not make silly mistakes (or only occasionally). Also, be able to state what you desire. Have any docs or materials in the same directory so the model can reference it. For even better results: turn on speech recognition and braindump what you know about the system, its goals, its context, history anything relevant, any gotchas you'd explain to a new employee or intern. Talk for 5-10 minutes. This is optional, "make it faster" can already get a large part of the job done.

And if it doesn't work well, describe what you dislike in its solution and tell it. Even just one extra iteration can make things work.

(I guess GPT-5.6 can be similarly good, I use Claude)

I feel like some people are emotionally invested in it not working and subconsciously sabotage their own effective use of the tool.

▲

qsort 2 hours ago | parent [-]

I'm not saying I don't get any use out of them, fwiw. In a lot of ways they write perfectly fine code, I'm just saying without some guidance at the macro-level they lose the forest for the trees, that's all.

	▲	bonoboTP 2 hours ago \| parent [-]
		Yes, definitely. It's too happy to just go and diligently work based on too little information, instead of asking questions first. And it makes many people think that they don't have to specify what they actually want, and the AI will mindread them and do what they want without being given access to docs and tools and context info that's crucial to know and they would intuitively know to hand over to a human collaborator but with AI they think they don't have to.

▲

sigmoid10 4 hours ago | parent | prev [-]

Which model and harness did you try?

	▲	qsort 2 hours ago \| parent [-]
		5.6 Sol on Codex, Opus 5 on Claude Code. (See sibling message, I'm not saying they're useless)