There's some good ideas and points in here, but this bit threw me:

> # We tried this > In July 2025 we went full lights-off

Isn't it pretty well-accepted at this point that the models underwent a step-change in usefulness around fall 2025 / spring 2026? I know that I was able to start handing agents whole features after that, but not before.

I feel like any perspective/experience on "what agents can/can't do" from before that period is... maybe less than relevant to the modern era. TFA calls it out a few sections later with "But surely the models have gotten better since then", but then just writes off any improvement. That does not match my experience.

I just listened to a podcast last week with the author (Dex Horthy) on as the guest. Humanlayer (his company) is constantly trying to push the limits with agents and he's very into using them.

He's well aware of exactly how good the models are today and I'm sure that if he thought they were good enough now to make the dark factory idea work, they'd be trying again.

I had a bit of this impression when reading the post as well as the authors' product website. A lot of it does seem to be stuck in 2025.

For instance I think their post "long-context isn't the answer" on their website post straight-up isn't accurate, and gives me the impression they are just extrapolating previous performance to new models. In my experience, Opus 4.6 and newer have worked very reliably for long context to me (i don't perceive any intelligence drop at 700-900K tokens). Yeah it's extremely cost-inefficient, but it works.

Opus 4.5 was a massive step change in capability for sure. The opinions of anyone who hasn't bothered experimenting with these things post-Opus-4.5 are not worth a damn.

They still are far from perfect but they're massively helpful.

The obvious statement would then be: 2025 we went all in, 2026 still nothing improved substantially. I also agree with that statement, none of the frontier models handle drift any better. Shotgun surgery and all that. You're waving him off by waving him off, but provide no arguments whatsoever. Is your experience substantially different?

my perhaps controversial take is that opus 4.1 was smarter than 4.5 for complex engineering work, but 4.5 was faster and "squishier" - it responded better to simpler prompts, it read between the lines of user input better, and that this was really important to converting new users quickly

Did you even read further on? The section right after that addresses the step changes in current frontier models https://github.com/humanlayer/advanced-context-engineering-f...