if the ai is writing everything wouldn't it be better to choose a more popular language?

benzible 3 hours ago | parent | next [-]

Another BEAM-based language, Elixir, came in first in a recent agentic coding study: https://dashbit.co/blog/why-elixir-best-language-for-ai

▲

sczi 2 hours ago | parent [-]

In that Tencent study less common languages got easier problems. Which explains why racket also performed suspiciously well in it. The authors recognize the problem themselves. I like elixir, but it's imo embarrassing how often elixir people cite that study despite it not actually supporting what they claim.

A much more recent and imo more realistic comparison of different languages with ai, is openai's study reimplementing sqlite in different languages: https://openai.com/index/low-resource-programming-languages/

That showed about what we expect, that popular languages like python, javascript, java, C#, etc do best, while less popular languages like Common Lisp, Ocaml, Zig, etc do slightly worse. But the difference is small enough I'd still just choose whatever language and ecosystem I know best that is appropriate for a project.

▲

yladiz 15 minutes ago | parent [-]

I would definitely be skeptical of a study that finds using an LLM/agent to write sqlite in JS being better than writing in Zig, but it strongly depends on what “better” or “worse” mean.

	▲	sczi 5 minutes ago \| parent [-]
		It means how much of the sqlite test suite it passed. They were just measuring correctness, not performance or memory consumption, where of course Zig should win. But the aim of that experiment was measuring correctness of llm written code in different languages, so used sqlite as a benchmark considering it's fairly complex software with a comprehensive test suite. It is not suggesting that because a language scores better on that benchmark it'd be better suited to writing the next sqlite, due to obvious other factors like performance...

▲

ModernMech 2 hours ago | parent | prev | next [-]

Exactly the opposite -- in a lot of ways, mainstream languages represent a compromise that gives the worst of all worlds for the benefit of popularity. With AI, you can take advantage of unpopular esoteric languages that have measurable advantages over mainstream languages. Before, the choice for a mainstream language was too strong because of network effects. Now with AI those effects don't matter as much, so being mainstream isn't as much of an advantage.

The copious training data on mainstream languages doesn't really translate as a network effect, and therefore is kind of a red herring because the AI can use all that training on new languages as well. e.g. if you use a esoteric language that is vaguely object oriented, the AI will use its vast training on object oriented programs and apply it to your language instead of falling over because your language isn't Python and it doesn't have training data on it.

Moreover, the AI is trained on copious amounts of PL research, so it's not like the ideas in a new language are completely new to it.

▲

lpil 7 hours ago | parent | prev [-]

Why's that?

▲

tharmas 5 hours ago | parent [-]

Lots and lots of production code to train AI models.

▲

giancarlostoro 3 hours ago | parent | next [-]

That mindset is a bit dated when models can pull up docs for a few years now. You can even have it write itself a little guide for lesser known languages within its own agents file.

▲

perrygeo 3 hours ago | parent | prev | next [-]

The entire gleam language tour fits comfortably in a modern LLM context window. You're one web search away from "training" it yourself.

▲

lpil 5 hours ago | parent | prev | next [-]

Programming models have been sophisticated enough for some time to not be restricted to specific languages. It's rather like the difference between a junior developer (who knows how to use a specific language) and a mid developer (who knows who to program generally).

	▲	jeremyjh 3 hours ago \| parent [-]
		Thanks for creating Gleam! Is there any research that supports what you are saying here? I agree it mostly works, but I’d expect more API hallucinations and test cycles at a minimum.

▲

ch4s3 3 hours ago | parent | prev [-]

I'm not sure how much that matters. I suspect and anecdotally hear/see that compiler errors and docs are as/more important.