One major disadvantage here is the lack of training data on a "new" language, even if it's more efficient. At least in the short term, this means needing to teach the LLM your language in the context window.

I've spent a good bit of time exploring this space in the context of web frameworks and templating languages. One technique that's been highly effective is starting with a _very_ minimal language with only the most basic concepts. Describe that to the LLM, ask it to solve a small scale problem (which the language is likely not yet capable of doing), and see what kinds of APIs or syntax it hallucinates. Then add that to your language, and repeat. Obviously there's room for adjustment along the way, but we've found this process is able to cut many many lines from the system prompts that are otherwise needed to explain new syntax styles to the LLM.