Why have the LLMs „learned“ to write PRs (and other stuff) this way? This style was definitely not mainstream on Github (or Reddit) pre-LLMs, was it?

It’s strange how AI style is so easy to spot. If LLMs just follow the style that they encountered most frequently during training, wouldn’t that mean that their style would be especially hard to spot?

For this "LLM style were already the most popular, that's how LLM works, then how come LLM style is so weird and annoying" I have 2 theories.

First, LLM style did not even exist, it's a match of several different styles, choice of words and phrases.

Second, LLM has turned a slight plurality into a 100% exclusivity.

Say, there are 20 different choices to say the same thing. They are more or less evenly distributed, one of them is a slightly more common. LLM chooses the most common one. This means that

   situation before : 20 options,  5% frequency each
   situation now    :  1 option, 100% frequency

I think these 2 theories explain how can LLM both sound bad, and "be the most common stye, how humans have always talked" (it isn't).

Also, if the second theory is true, that is, LLM style is not very frequent among humans, that means that if you see someone on the internet that talks like an LLM, he probably is one.

This is total speculation, but my guess is that human reviewers of AI-written text (whether code or natural language) are more likely to think that the text with emoji check marks, or dart-targets, or whatever, are correct. (My understanding is that many of these models are fine-tuned using humans who manually review their outputs.) In other words, LLMs were inadvertently trained to seem correct, and a little message that says "Boom! Task complete! How else may I help?" subconsciously leads you to think it's correct.

You may thank millenial hipsters who used think emojis are cute and proliferation of little javascript libraries authored by them on your friendly neighborhood githubs.

Later the cutest of the emojis paved their way into templates used by bots and tools, and it exploded like colorful vomit confetti all over the internets.

When I see this emojiful text, my first association is not with an LLM, but with a lumberjack-bearded hipster wearing thick-framed fake glasses and tight garish clothes, rolling on a segway or an equivalent machine while sipping a soy latte.

LLMs write things in a certain style because that's how the base models are fine tuned before being given to the public.

It's not because they can't write PRs indistinguishable from humans, or can't write code without Emojis. It's because they don't want to freak out the general public so they have essentially poisoned the models to stave off regulation a little bit longer.

My impression is that this style started with apple products. I remember distinctly opening a terminal and many command lines (mostly Javascript frameworks) applications were showing emoji in the terminal way before LLMs.

But maybe it originated somewhere else.. In Javascript libraries..?

I do remember 1 example of an emoji in tech docs before all of this -- learning github actions (which based on my blog happened in 2021 for me, before ChatGPT release), at one point they had an apple emoji at the final stage saying "done". (I am sure there are others, I just do not remember them.)

But agree excessive emoji's, tables of things, and just being overly verbose are tells for me anymore.

I wonder if it's due to emojis being able to express a large amount of infomation per token. For instance, the bulls-eye emoji is 16 bits. Also, Emoji's don't have the language barrier.

I wonder if there's an analogy to the style of Nigerian e-mail scams, that always contain spelling errors, and conclude with "God Bless." If the writing looks too literate, people might actually read and critique it.

God Bless.

There's some research that shows that LLMs finetuned to write malicious code (with security vulnerabilities) also becomes more malicious (including claiming that Hitler is a role model).

So it's entirely possible that training in one area (eg: Reddit discourse) might influence other areas (such as PRs)

https://arxiv.org/html/2502.17424v1

> Why have the LLMs „learned“ to write PRs (and other stuff) this way?

They didn't learn how to write PRs. They "learned" how to write text.

Just like generic images coming out of OpenAI have the same style and yellow tint, so does text. It averages down to a basic tiktok/threads/whatever comment.

Plus whatever bias training sets and methodology introduced

It reminds me of this, but without the logic and structure: https://gitmoji.dev/

I'm glad that AI slop is detectable. So, for now the repulsive emoji crap is a useful heuristic to me that someone is wasting my time. In a few years once it is harder to detect I expect I'm going to have a harder and more frustrating time. For this reason I hope people don't start altering their prompts to make them harder to detect as LLM generated to people with a modicum of intelligence left.

Don't Github have emoji reactions? I would assume that those tie "PR" and "needs emojis" closely together.

RLHF and system prompt, I assume. But isn't being able to identify LLM output a good thing?