One of my concern is LLMs are going to generate a lot of low quality code for languages that do not have sufficient discussions on forums like Stackoverflow.

That's why these niche languages need state-of-the-art compilers that enforce invariants more strongly. This way, they can catch most of the subtle bugs the LLM produces, sort of like antibodies.