They seem more marginal because the models are already capable of so much. But there was a turning point about November of last year if I recall where software developers I know went from "lol look at how this thing codes, my job is safe" to "oh no". And the models back then were actually pretty bad at many things compared to Astra or Opus 5.5.

You're right that many of the less capable / open weights models are very good themselves, but that's part of showing how much more capable the frontier models are becoming. Cheaper (smaller) models are able to do the same things, in many cases better, as the way larger models were able to do a year ago. The efficiency is getting way better alongside the 'intelligence'. That'll also cause the models to become much faster over time which is the real improvement I'm waiting for.

I think a part of why many people aren't seeing big deltas if they're using it for code and they're actually coders is we actually sit down and reason out the project with the LLM. Someone who knows nothing about coding ~1 year ago wouldn't come even close to the kind of app they'd get now with the same instructions. If you just give it a task like "help me decompile this game" without knowing a whole lot about decompiling suddenly in the last 3 or so months you start to get some half-decent results.

The sea-change I see lately is about lifting up the people who couldn't do these things before more than the people who could. And I actually see problems coming from that already as people I've met think they can make enterprise grade software if they just tell their LLM to make it safe.