LLMs can definitely write programming languages. I used Gemini CLI (once it came, AI Studio before that) with Gemini 2.5 Pro to create one with Rust and LLVM:

https://github.com/GoogleCloudPlatform/aether

This was completely vibe coded - I never had to edit the code, though it was very interactive. The whole thing tool less than a month of some evenings and weekends.

(Note: it’s ugly on purpose, as I’m playing with ideas around languages that LLMs would naturally be effective using.)