Remix.run Logo
anonymous908213 3 hours ago

It is not based on guesswork. For whatever it's worth, I have gotten 7 LLM accounts banned from HN in the past week based on accurately detecting and reporting them to moderation[1]. Many of these accounts had between dozens and 100 upvotes, some with posts rated to the top of their threads that escaped detection by others. I have not once misidentified and reported an account that was genuinely human. I am aware that other people have poorly-tuned heuristics and make false accusations, but it is possible to build the skill to detect LLM output reliably, and I have done so. In the end, it is up to you whether you believe me, but I am simply trying to offer a warning for people who dislike reading generated material, nothing more.

[1] Unlike LLM-generated articles, posting LLM-generated comments is actually against the rules.

gchamonlive 2 hours ago | parent | next [-]

Congrats, and thanks for your work, but you should be aware that HN comments are completely different from articles. What makes you think the skills/automations required to identify LLM generated HN comments will work seamlessly with submitted articles? You have to do a statistical analysis of this, otherwise it's just guesswork.

You also have to take into account that the medium is the message[1]. In a nutshell, the more people read LLM generated posts and interact with chatbots, the higher the influence of LLM style in their writing -- the whole "delve" comes to mind, and double dashes. So even if you have a machine that correctly identified LLM generated posts, you can't be sure it'll keep working.

[1] https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf

famouswaffles 2 hours ago | parent | prev [-]

Those are a lot of words to say you guessed. And the banning comment is nice I guess but pretty meaningless. Does moderation really always report back to you when you make such an accusation ? Who's to even say all the banned accounts were LLMs ? You know what would happened if i got banned because someone accused me of being a LLM ? Nothing. I'd take it as a sign to do other things.

Let's say you are the LLM detecting genius you paint yourself to be. Well guess what? You're human and you're going to make mistakes, if you haven't made a bunch of them already. So if you have nothing better to add to a post than to guess this, you probably shouldn't say anything at all. Like you said, it's not even against the rules.