They are getting better at historical data, not at the fundamental issue.

As a recent example, I recently had to abandon the multiple LLM reviewer/verifier model I was using because zig 0.16 was released with major changes.

I actually reverted back to full self hosted because the foundation models we’re trying too hard to revert to the older versions of the language.

It is going to be a balancing act and there is fundamentally no way for LLMs to get around this.

We will have to develop methods to do so, most likely by focusing agents on problems that are more static.

Question for you, since I also use Zig 0.16: how do you get it to use Zig idioms? I use Kimi 2.6, and I feel like whenever I try to get my agent to write modern Zig based on a C reference it decides to start writing everything in a C style (doesn't use defer, doesn't use opaque enums even when I explicitly tell it to, doesn't use Zig's error unions, swallows errors instead of asserting, and some more). It's quite frustrating, and a lot of catchable errors crop up until I've beat modern practices into it.

I find great success in not relying on LLM's built-in knowledge, but giving it links to necessary docs/manuals and have it read that before doing anything.