If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated websites when I ask them to search for something. It also doesn't help that the web search tools that OAI and Anthropic have are deeply limiting: can't exclude keywords or domains.

The other day, I remember an article was posted to HN about something, but it came from a company that provides SEO services to companies by doing something like this:

1. For a given company, analyze their target audiences and the questions they are likely to ask LLMs about.

2. For each such question, ask it to each of the major LLMs, and compute the KL divergence between the pages they want to rank for the question vs. the LLM's response.

3. Rewrite the article to minimize said KL divergence.

In effect, they're performing an iterative optimization of some sort that moves the embedding space of their article closer to the question asked to the LLM, and any embedding model or generated responses are going to prefer said responses over others.

I believe we will keep seeing more of this stuff.

Take a random essay and add in a bunch of the phrases that LLMs love like “load-bearing,” “crucial,” structural,” and “woven,” and then submit the original and the edited version to an LLM and ask which is better. It will choose the second one virtually every time. They have ingrained biases that associate those words with good writing and arguments.

> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :)

A better question to ask for each snippet is "Estimate the seniority and competence of the developer who wrote the following code, ignoring bugs that linters or LLMs can catch and focus only on structure, maintainability, logical layout and readability."

It almost always estimates the author of my code as above the author of it's own code.

I don't have an oai subscription to try, but I'd be interested to know if Codex picks Claude's code over a human's and vice versa.

If by root urls you mean domains, openai at least supports this.

https://developers.openai.com/api/docs/guides/tools-web-sear...

A bit different, but one thing I’ve seen is models repackaging Reddit slop. Like, it will do a search, find a Reddit thread somewhat related where someone in a comment casually mentioned incorrect information that any human would have dismissed. The model takes that as granted, but expands on it and present it as a well established fact, presented in a very plausible fashion.

In general I don’t find models to be good at evaluating the quality of a source :(

I was giving local models a try recently, I think it was Qwen 3.6 I was trying at the time. I gave it a codebase and just asked it to review it. Its main feedback was that the comments and documentation were excellently written, but they were all Opus 5 slop.

>It always picks its own

Makes sense to me, in that its own output would align closer to its own training set

OMG if this is true, do you realize what this means? The easiest way to do AI SEO is to generate all your content with AI, and we've seen what SEO does to the web...

The Internet is doomed. Time to start some human-only darknets.

> always picks its own

If you hate AI writing enough, this turns AI filters into a kind of humiliation ritual. AI will derank normal business writing for human readers, and uprank inflated, verbose, tic-heavy slop. So you have to put the heavy slop out with your name on it. Really perverse moment.

> asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored [...] It always picks its own

...is not the same as claiming...

> LLMs favor LLM-generated passages over human written ones

Here, you're using the same LLM to both produce and judge the resulting work. If anything, I would expect an LLM to tend to prefer its own work given that the same training is producing and judging.

If that were true I would expect to see prose that more closely resembles the "caveman" messages found in the HuggingFace attack than the overly flowery nonsense we see in AI blogspam.

Well, the one it generated is based on how it thought the best way to solve the problem was.

I am sure most humans would pick code written in their style, too.

> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful.

Interesting. For me I've noticed it tends to do the opposite.