Remix.run Logo
meowface a day ago

This article is written by an LLM, by the way. ("Quietly" is... as Claude might put it... "often the quiet tell".)

embedding-shape a day ago | parent | next [-]

Which, given the section below from the article, ends up kind of ironic:

> There is a deeper cost still, and it concerns the thing LLMs do most impressively: writing. [...] Outsource the writing and you have not accelerated the thinking; you have skipped it.

I think the author might not know what "impressive writing" or even "good writing" is, which would explain both how they could put that part into the article, and how they seemingly believed this post was good and valuable enough to be posted publicly.

wgd a day ago | parent | next [-]

> the author might not know what "impressive writing" or even "good writing" is

Well the author in this case is Claude, and AIs write like that because the assistant persona really thinks that's what good writing sounds like. They're wrong, but they're just doing what they were taught. There's a reason LLM raters consistently score LLM writing highly.

grey-area a day ago | parent | prev | next [-]

Well perhaps the LLM put that bit in for them.

bobajeff a day ago | parent | prev | next [-]

Interestingly that's the one point of the article I have a disagreement with. Yeah, good thinking comes when reformulating your ideas. Also, reformulating ideas is part of the traditional writing process. However it's does not necessarily focus writers/thinkers on the reformulating ideas in their most value adding form.

I've watched some YouTube videos narrated by AI that were created by those who's native language I assume to be Chinese but the value of the content is higher and more concentrated than those from the native speakers.

While I've watched many for whom English is not their first language struggle in technical talks and lose most of the meat if their discussion to their struggle with the language conversion.

While the example of language barriers being skipped over and providing value in that circumstance is obvious I suspect more is possible by avoiding unnecessarily focus on prose or technical aspects of communication and focusing on the ideas themselves.

Imagine if the cost of not assuming a background in technical/textbook writing were zero. Freeing up authors to explain more thoroughly. Perhaps, more effort can be spent on thinking up analogies, metaphor or examples to help communicate an idea.

ModernMech a day ago | parent | prev [-]

> believed this post was good and valuable enough to be posted publicly.

I mean, it made it to the front page of HN didn't it? Probably served its purpose just fine. Not all writing is supposed to be impressive or good. Some of it is to just get attention and stir discussion, and this Claude output did exactly that.

scronkfinkle a day ago | parent | prev | next [-]

I wanted you to be wrong, and to be able to make this an example of us over-reacting to certain trigger words created by AI, but unfortunately I just scanned the first couple paragraphs with pangram and it reported 100% AI, so you're probably correct.

I get a funny feeling in my stomach over the idea that common and effective means of communication (i.e. it's not X it's Y) have become faux pas to use because of AI. I think it's something about these phrases being taken away from us more-so than the AI inventing them.

zamadatix a day ago | parent | next [-]

Pangram should be paying HNers for how often we pitch needing to use their product by name to believe things as obvious as "a long form news article cramming every AI trope it can fit from start to finish" being AI written.

At this point I'm surprised when a news article isn't largely AI written, let alone one using the default tone! I don't even mind it as much as others seem to, it's just turning into more and more of a rarity for a news article to not be these last few years and so is now what sticks out.

dwedge a day ago | parent | next [-]

> Pangram should be paying HNers for how often we pitch needing to use their product by name

I kind of assumed it is, given how it suddenly seemed to start getting namedropped in multiple comment threads. And it's often in response to someone saying content is obviously LLM (with examples) and the shill wedges pangram into the conversation "omg you're right, I didn't believe you but I checked this new product and wow it agreed with you" as though that adds anything to the discussion at all.

wgd a day ago | parent | next [-]

Pangram gets brought up a lot because if I read an article and think it's blatant AI slop and want to communicate that fact, a natural impulse is to provide some sort of objective corroboration rather than just asserting that I have superior taste and thus am able to tell.

Also Pangram is basically the only AI detector that's actually put in the work to build an accurate classifier, so if you try to discuss AI writing detection without being specific that you mean Pangram you'll get half a dozen commenters screaming about how awful some of the snake-oil salesmen like GPTZero are.

arjie a day ago | parent | prev [-]

I mention pangram because it’s good at what it does. Otherwise, people like to claim “That’s just how I’ve always written. I always say That’s the seam and the seam undercuts the other pillar. It’s not just correct, it’s also verified lol; it’s just how I speak” and it’s obvious bullshit. Depending on the politics of the poster, people will also support this kind of garbage, but it’s all drivel.

The authors are so often dishonest that it’s hard to believe that the things they’re saying are anything but a random hallucination.

wrsh07 a day ago | parent | prev [-]

You have to name pangram because the other ai detectors are quite bad

bee_rider a day ago | parent | prev | next [-]

Em-dashes would be a loss, they can be a better flowing version of a parenthetical. “It’s not X it’s Y” is not a loss. This phrase is a symptom of a situation where the author wants to subvert expectations but doesn’t have the space, ability, or faith in their audience to organically set up X as the thing to be contrasted against.

I suspect it became an LLM tell because it is over-represented in text that’s easily available to the models but that most people don’t actually want to consume: marketing text, LinkedIn posts, that sort of thing.

mlinhares a day ago | parent | prev | next [-]

That’s not common and effective, it’s buzz feed style articles.

teiferer a day ago | parent | prev | next [-]

Have you also scanned earlier work of the author with pangram to establish a baseline? Pangram is not without problems.

meowface a day ago | parent | next [-]

It is actually pretty much without problems. It doesn't catch everything, but if it detects something it's very rarely a false positive.

no_multitudes a day ago | parent | prev [-]

Can you give some examples of verifiable pangram false positives?

PunchTornado a day ago | parent | prev | next [-]

Why trust pangram? What are their accuracy scores in the wild? Ai detection is notoriously difficult.

no_multitudes a day ago | parent [-]

AI detection in long-form content is a problem well-suited to training a classifier model. We have tons of verifiably-not-AI text from before 2022, and you can create tons of verifiably-AI text. As language drifts over the next few decades, it may get more difficult. But right now it is quite a manageable problem for languages with large pre-2022 text corpora available online.

The reason why AI detection tools other than Pangram are awful is because they are not really trying to solve the problem -- they just want to appear good enough to convince people to use them.

nicce a day ago | parent | prev | next [-]

All these "is this AI" checkers should be banned, as they never can reach 100% accuracy.

wrsh07 a day ago | parent | next [-]

This is a truely bizarre claim. Imagine if one said:

"Weather forecasts should be banned because they can never reach 100% accuracy"

nicce a day ago | parent | next [-]

They are completely different things with different consequences. Someone can point to the sky and prove that the sun is there.

joshmoody24 a day ago | parent [-]

You're responding to a comment about weather forecasts, not current weather. Pointing to the sun is not a forecast.

PunchTornado a day ago | parent | prev [-]

Imagine writing an article in 5 days. Working hard. Posting it online and then everyone says it is ai and laughs and dismisses it because a tool says it is AI.

Matumio a day ago | parent | next [-]

Like in this comic about art: https://www.peppercarrot.com/en/miniFantasyTheater/064.html

wrsh07 a day ago | parent | prev [-]

Imagine wasting several thousand people's time

teiferer a day ago | parent | prev [-]

They are overrepresented in HN discussions for sure as there is an over-reliance on them that just aims to shut down discussions.

Which is ironic since they are a tool folks are trusting blindly supposedly used to argue against tool use with blind trust.

wrsh07 a day ago | parent | prev [-]

Fwiw, I only did the first 300 words (signed into pangram) and it seemingly correctly noted that there were 2 (mostly) human-authored sentences in there

What's wild to me is that:

1. People are responding to this article like it's hitting a nerve

2. In most Ubers you already don't talk to the driver (yellow cabs are higher variance in NYC). Doesn't seem like anything's being lost in that case.

3. There are enormous safety benefits to waymo, mobility benefits for youth (and elderly) that are afforded by this technology. It's not clear why people argue "Uber" is better than waymo. A few years ago there were arguments against Uber! (A technology which, again, provides a huge benefit, especially if you live in an area where people were previously expected to go out for drinks and then drive home)

I think it's fair to point out real issues at these companies. But we should be clear-eyed about which technologies we want to accelerate vs slow down

dwaltrip a day ago | parent | next [-]

It's super obviously AI-written. I can smell this shit from a mile away now, after using Claude Code so much.

In the just first few sentences I could tell.

teiferer a day ago | parent | prev [-]

Have you actually read the article? They point out multiple times that they like the Waymo ride and would use one again and loved it. It's not a Waymo-bashing article, it cautions against the long term effects on research.

And here you have it — I used an LLM-ism. Oh, and now another one! Time to downvote me for supposed LLM use! (Which I obviously didn't, I'm typing this on an ancient smartphone waiting for a train, but that doesn't deter the witch hunters.)

meowface a day ago | parent | next [-]

You actually didn't. The construct is slightly different.

I use LLMs all day every day. I think they're great. I just don't like humans who put their name to what an LLM did.

teiferer a day ago | parent [-]

> You actually didn't. The construct is slightly different.

Well I know that I did, so you are wrong with that claim, which sheds a strong (negative) light on other statements of fact that you posted in this discussion.

You may disagree with me, or I may even have gotten something wrong, but just claiming I didn't read the article ... sorry man, I thought HN had higher standards than that.

no_multitudes a day ago | parent | next [-]

They were pointing out that "It's not a Waymo-bashing article, it cautions against the long term effects on research." does not sound like LLM-generated text.

It would sound kind of LLM-y if you had said "It's not an anti-Waymo article. It's an article about how assistive technology is quietly degrading research."

(Of course, the correct approach to detecting AI is not counting LLM-isms, but feeding long-form text through a classifier model and picking up statistical correlations that are more in-distribution with LLM text than human text.)

meowface 20 hours ago | parent [-]

Correct. I was already pretty sure, but I wouldn't have written that post without checking Pangram first.

meowface 20 hours ago | parent | prev | next [-]

"It's not a Waymo-bashing article, it cautions against the long term effects on research." is not an LLMism. It superficially resembles one, but that's it.

(Nor is what I just wrote, there.)

wrsh07 a day ago | parent | prev [-]

I think they were referring to your fake llmism

teiferer a day ago | parent [-]

Ah thanks. Fair.

sdthjbvuiiijbb a day ago | parent | prev [-]

Calling it "witch hunting" is just dishonest. That term implies a baseless overreaction to a heavily exaggerated or imagined threat.

That's hardly what's going on here. Unlike witches, AI slop really is everywhere and it's pretty clear that this article is slop.

I find complaining about AI callouts to be itself deeply distasteful although it would take some thought to put my finger on the exact reasons why.

teiferer a day ago | parent [-]

Yes AI slop is prevalent. But the casualties that get thrown under the bus are also real, don't you agree? Is it really worth it to sacrifice those? In the name of the larger good?

I'm arguing that we're losing something by doing so, as a community.

> I find complaining about AI callouts

If they are correct then I'm all for it. But just suspicions turned into statements of fact are not helping anybody. People used em dashes before LLMs Not nearly as much as LLMs, but superficially judging people isn't doing any good.

I'd compare this with pitchforks coming out agains criminal immigrants. Yes, some immigrants are criminal. Doesn't mean that if you stand in central Copenhagen and have a dark skinned person in front of you that it's a criminal.

wrsh07 a day ago | parent [-]

The rule of thumb is simple: if most of a user or author's writing is consistently human-generated, then I think we are very happy giving them the benefit of the doubt if one article or snippet flags the actually good ai detector

Unfortunately, many of the largest voices against pangram simply don't like it because it gives their lies less credibility.

I don't mind reading llm writing, indeed, I read more llm writing per day than most. But if you're using LLMs to increase the level of slop (blog posts, comments, tweets, etc) then we should call people out.

It's a colossal waste of everyone's time and attention, and we should be mad about it.

Late edit: if people are actually writing and it's consistently flagged by pangram (this is a statement I have not yet seen validated), the pangram folks are extremely proactive and excited to understand what's going on. This notion that somehow the authors of slop are victims is complete nonsense.

teiferer a day ago | parent [-]

> if most of a user or author's writing is consistently human-generated, then I think we are very happy giving them the benefit of the doubt if one article or snippet flags the actually good ai detector

I would agree to that approach.

But that didn't happen here. Nobody has done that due diligence and folks are just blindly accusing. The author has published for many years. That's what I'm calling out.

> This notion that somehow the authors of slop are victims is complete nonsense.

Strawman? I didn't claim that or anything close to it. I also find the prevalence of slop writing hugely annoying.

dash2 a day ago | parent | prev | next [-]

Is it OK to flag articles for being AI-written? I really want to.

righthand a day ago | parent | next [-]

Yes, HN is for human discussion. It is fine to flag and remove things that are not human written and cannot facilitate human discussion due to it’s false construction.

internet2000 a day ago | parent | prev [-]

No, it is not.

happytoexplain a day ago | parent | prev | next [-]

Is "quietly" an LLM tell? It was always certainly a human writer trope commonly seen in journalism. Though I guess it does appear five times in the body of the post, and a human writer would probably not go that far with it.

embedding-shape a day ago | parent | next [-]

I think individually and when used sparingly, writing tropes can be fine, but when the article has every second sentence being a writing trope, it becomes pretty obvious that either a LLM wrote it, or the author simply doesn't know how to write, and regardless, it's a waste of time trying to read through it.

meowface 20 hours ago | parent | prev | next [-]

In this sort of way, yes, it's a clue. Obviously not dispositive just by itself.

duskdozer a day ago | parent | prev | next [-]

It genuinely is--and that's the honest truth.

AlexandrB a day ago | parent | prev [-]

The real LLM tell is tortured and inappropriate metaphors and unnecessary adjectives and adverbs[1]. What purpose is "quietly" serving in this headline?

[1] https://www.youtube.com/watch?v=ORgKY9AlybA

Oarch a day ago | parent | prev | next [-]

You're absolutely right—it's been a load-bearing LLM tell for a while now.

nicce a day ago | parent | prev | next [-]

I think that in the end it does not matter, anymore. It is inevitable that most of the text now and in the future is at least reviewed by LLM.

We should criticize whether text is poorly written, inaccurate, wastes words to get to the point and anything like. Saying that it is "written by LLM" just bypasses this and is not useful and misses the point. AI written text can be really good if used correctly. Criticize the content, don't speculate how it was written. If AI helps us to write better text, that is great. But often it is not good.

It is better blame the author for the bad text, so they get consequences and might do better next time, if the text is bad.

By the way, I think this piece of text was quite good.

ripe a day ago | parent [-]

> saying it is "written by LLM" is not useful and misses the point.

It is useful to me. I see too much slop every day and would like to filter it out from my life. I appreciate the tip, and to me, that is the whole point.

teiferer a day ago | parent | prev | next [-]

Do you have more evidence than this? I'd honestly like to know.

I don't like AI slop like everybody else, but I'm also growing skeptical of the very fast determinations that sth is supposedly made by LLM just because it uses some phrase that Claude also uses. Models are trained on text written by human writers and if human text resembles it, this goes both ways. I have looked at the authors work from the pre-AI era and it reads similar to me.

So please, if you come with such a claim, please include what you base it on, so we all have a chance to determine how much to trust your verdict.

ricksunny a day ago | parent [-]

"every hour spent aligning with a co-author is an hour not spent producing output that is legible to an evaluation system." use of the word 'legible'.

"We defund the corridor and then wonder where the corridor conversations went." this is a claude-ist construction.

"This is an old worry wearing new clothes" this is a very claude-ist construction.

(rather ironically... as @embedding-shaep pointed out earlier): "Outsource the writing and you have not accelerated the thinking; you have skipped it." 'not X but punchy-Y' (not to be confused with 'not X but profound-Y'.

I'm not saying the author shouldn't have used an LLM - once in a while I suppose it gets used to good effect. But I wouldn't kid myself that the writing is certainly all human through-and-through.

[Incidentally, in case anyone's wondering where many of these claude-isms come from, just search within lesswrong.com .]

s08148692 a day ago | parent | prev | next [-]

And I read it by asking an LLM to summarise it

bayindirh a day ago | parent [-]

Can you share me the summary? I'd like to read a LLM summary of your summary.

Things are bit tight today.

NotSuspicious a day ago | parent [-]

[flagged]

iamacyborg a day ago | parent | prev | next [-]

", and" is the more obvious tell for Opus/Fable writing.

jackling a day ago | parent [-]

Plenty of people use Oxford commas though, can't really use it as a reliable signal, too much noise.

mobilejdral a day ago | parent | prev | next [-]

This comment was perhaps written by an LLM that has learned it can karma farm by pointing out all of the articles that have LLM usage while everyone still thinks that is a novel contribution.

meowface a day ago | parent | next [-]

If you check my comment history here, I have not written this before.

I like LLMs and use them daily, but I don't really like when people use them undisclosed for long-form prose writing.

teiferer a day ago | parent [-]

Me neither, but just running around accusing random folks of things is not going to help. To the contrary. If a real author doing real work with a lot of effort risks being accused of things they didn't do and just dismissed, you'll get less of genuine content, not more.

meowface a day ago | parent | next [-]

I will donate $100 to charity if I made a false accusation.

(It's possible AI rewrote an article first written by a human here or whatever, but the point is that I can just tell this is mostly or entirely LLM output.)

teiferer a day ago | parent [-]

> I will donate $100 to charity if I made a false accusation

That's honorable but not how accusations work. There is a first level indirect effect of painting somebody in a light that damages the reputation of their work or their person which isn't reversed by a donation or a sorry or anything since retractions/corrections/apologies don't get the same public attention. And there is a broader effect of chilling what people do since they don't want to get falsely accused.

I'd recommend keeping such suspicions to oneself unless there is unrefutable proof, and if there is, then please provide it.

done_lurking a day ago | parent | prev | next [-]

It does help, if the accusation is true. The fact that false accusations result in less genuine content implies that true accusations result in less disingenuous content. For better or for worse, social consequences do affect people's behavior on average.

teiferer 11 hours ago | parent [-]

All true. But question is whether the positive consequences outweigh the negative. That may be the case, but it may not be. I don't find it obvious and not considering this is careless.

dwaltrip a day ago | parent | prev [-]

The first few sentences alone are pure Claudish slop (I didn’t read any further). The stench is unmistakable once you’ve looked at it enough.

I’d also donate $100 if I’m wrong. I feel quite confident.

teiferer 11 hours ago | parent [-]

Yeah, lots of redheads were burned back in the day because people "felt" things.

dwaltrip 7 hours ago | parent [-]

It’s a blog post… I’m not proposing we burn the author. You alright?

fidotron a day ago | parent | prev | next [-]

In the spirit of the "That's what she said" bot, you could train an LLM detector by accusing everything of being authored by an LLM and seeing which ones get voted up.

happytoexplain a day ago | parent | prev | next [-]

"Novel" is not a prerequisite for something to be worth pointing out. Lots of bad things happen repeatedly, and don't quickly become not worth caring about or knowing about. People have strong spirits and curious minds and it tends to take a long time to boil that frog out of them - generations, sometimes.

throwawaysleep a day ago | parent | prev [-]

People aren't treating it as a thoughtful contribution. It is more of a flag, like "spam".

cj a day ago | parent [-]

HN already has a feature for that. Just flag the article!

happytoexplain a day ago | parent [-]

I appreciate when people write things like, "this is an advertisement for the author's product, X." Why stay silent? If the commenter's accusation smells fishy, I'll read the article myself. Otherwise, they helped me.

cj a day ago | parent [-]

To me, it’s the equivalent of “HN is turning into Reddit” - it’s just a complaint without adding to the discussion.

joe_the_user 21 hours ago | parent | prev | next [-]

It's sad given the topic is interesting and significant.

AlexandrB a day ago | parent | prev | next [-]

This rings more and more true every day: https://samkriss.substack.com/p/if-you-let-ai-do-your-writin...

a day ago | parent | prev | next [-]
[deleted]
SirFatty a day ago | parent | prev [-]

and silently.