That sounds like Haskell's ST monad and yeah, that could work. These are very specific "escape hatches" though and typically people want more.

As far as I can tell, the good models will use whatever you give them. So it seems we should only give them language features that help humans understand and maintain what the model writes.

Amusingly, a friend had Claude write some BrainF*ck the other day. Non-trivial algorithm, and it made working code on the first try.