For me it's really hard to find a balance with LLMs. Even trying one to add .NET IR codegen to my compiler that already had a working C++, JS, and C# backend for it to use as a reference. It just kept doing really odd stuff and when we got to generics it had done such a bad job with the prior work it struggled to collect the information it needed from .NET to do generic calls correctly, and mid session decided to just make everything reflection based with no notice to me. It had a doc of the AST with notes on what the nodes mean and how different constructs are represented, it's a pretty typical ML family language. Plus a working C# backend to reference.

Other times it will one shot things near perfectly that I was expecting to just use as a reference. Demo pages for work I've done its so fast and gets good enough right out the door

Every time I've tried to use it to write something small in my compiler? :"( pain. It's the perfect intersection of looks good and covers 90% of cases fine but explodes far away because some subtle invariant is wrong. I can't always think of the invariants up front without writing the code myself

When did you try that? The rate of improvement of models has been hard to keep up with.

I recently asked Gemini 4 Argon a question about a gcloud api, and to answer the question it decompiled the Google Java client library (without me asking) and figured out how it worked.

I do wonder if models struggle with .NET IR because there may be less info about it in its training data. When you’re working with open source systems it has much more to go on.

I’ve used models to generate ONNX IR (executable format for deep learning models), and that worked well. But the ONNX IR is much simpler.