It'd be interesting to see how well the LLM would be able to write code using the new language since it doesn't exist in the training data.

I've tested this, the LLM will tend to strongly pattern match to the closest language syntactically, so if your language is too divergent then you have continually remind it of your syntax or semantics. But if your language is just a skin for C or JavaScript then it'll do fine.