Remix.run Logo
2001zhaozhao 5 hours ago

I suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.

At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.

culi 5 hours ago | parent | next [-]

It's already been happening for a very long time. The US' "Operation Earnest Voice" happened in 2010/2011[0]

And it's not just state actors. Anyone who was online during the Amber Heard/Johnny Depp trials was likely inundated with media about that trial. Both of their legal teams utilized bots to drive online discussions[1][2]

[0] https://en.wikipedia.org/wiki/Operation_Earnest_Voice

[1] https://www.latimes.com/entertainment-arts/story/2022-07-19/...

[2] https://www.tortoisemedia.com/2024/02/26/depp-v-heard-who-tr...

CGMthrowaway 4 hours ago | parent [-]

Why are you being downvoting for sharing this relevant truth, complete with sources?

culi 4 hours ago | parent | next [-]

All threads that have... certain keywords... are like this. Talk about meta. Looks like I'm now in the positive (+5) after being in the negative just a moment ago.

Anyways, I got really interested in this topic around mid 2010s because of some really interesting reporting by Codastory[0] covering Russian "troll factories" especially used to influence opinion in eastern Ukraine, but also in many other circumstances. In Houston in 2016, Russian troll accounts planned both an ANTI Muslim rally AND a counterprotest to that rally all on Facebook.[1] Hundreds of real human beings showed up!

Russia got a ton of attention for their use of troll farms but the truth is that the US and certain allies have actually been using stuff like this for far longer and, imo, with much more success. In fact, Coda Story itself is funded by the NED which is a massive pillar of global US, uh, "soft power".

[0] https://www.codastory.com/

[1] https://www.npr.org/2017/11/01/561427876/how-russia-used-fac...

Paradigma11 2 hours ago | parent [-]

For me the main difference is if the activity is clandestine and meant to deceive/hide the origin. Everybody tries to influence things their way. Doing it covertly is a hostile act. That does not mean that I would not condone hostile acts against my/our enemies but it is something take into account. And regarding your Russian examples, it showed for how long they have seen the west as enemies and conducted hostile acts.

aa-jv an hour ago | parent | next [-]

They view certain components of "The West" as their enemy - indeed, the very organizations which conduct the same operations against the Russian people.

smalllangmodel an hour ago | parent | prev [-]

I wonder why they view the west as enemy, nothing ever happened to them right? :P

kakacik 40 minutes ago | parent [-]

Thats rather simple to understand, for somebody coming from eastern Europe. russian mentality permeated entire region during their enslavement of what is now eastern EU bloc for +-40 years, to have a battleground for nuclear war with west that never came. It happens when all your politics is orchestrated in moscow and you have almost a million soldiers deployed in bases all over the place for decades, with nuclear weapons and all that.

See, russians have massive inferiority complex. Against everybody, but mostly the more successful ones, meaning west in many aspects. Their approach to, well, everything is vastly ineffective, corruption, nepotism and similar stuff grinds any attempt at progress to almost halt. Just go there and see for yourself, they have money literally sticking out of ground left and right yet outside moscow and st petersburg you are in central Africa in terms of development.

All this, propagated to 21st century where everybody seems to go ahead of them. A lot of frustration, envy and schadenfreude. A (correct) feeling russia is left behind and becoming insignificant local force at best, and laughing stock of whole world more realistically. Plus, human life is generally almost worthless there, its mafia state pyramid to the core.

With all that, its vastly easier to keep reinventing enemies that are always guilty at damaging poor old russia, rather than cleaning up that clusterfuck and actually owning the shit they created for themselves. Its enough for general population. There is not much more actual stuff to the question 'why west', rest is just PR for simpler folks. A dictator never wants smart, strong and free population, double that for russia. Downtrodden, hopeless desperate folks are much easier to manage and manipulate.

smalllangmodel 20 minutes ago | parent [-]

Well, you could also look at the history and cold war and NATO and so on, but sure, the Orcs are just having inferiority complexes!

friendzis an hour ago | parent | prev | next [-]

Because cognitive biases are one of the hardest things to even acknowledge, let alone suppress. All the media out there has such a strong bubble effect, because people hate being challenged and love to stay in their comfy echo chambers. Remember labubu? Bao buns certainly do not.

There have been so many bot armies exposed over the years that one would think every narrative being pushed should be considered to be backed by bots. Some state-level actors even operate their narrative-pushing campaigns openly, yet people still happily deny their entire existence.

HN is very much not an exception to this rule. It's very hard to accept that maybe some thoughts you have were actually installed.

derefr an hour ago | parent [-]

> people still

Maybe a few are just “people” (i.e. the same people who normally post in the community.)

But in most cases I’d suspect any such botting campaign brings along with it a few human agitprop agents embedded onto each target platform, well-positioned to deny and discredit anyone pushing back on the narrative the bot farm is spreading.

(Sometimes this is pretty obvious — a lot of accounts coming out of the woodwork to deny Chinese bot farms have a conspicuous “English as learned as a second language in Beijing, to a competent but not fluent level” writing style that does not often otherwise crop up in other discussions in those same communities. But other times it can be far more subtle, making use of operatives hired from within the community’s own majority-culture rather than relying on foreign nationals playing the part of same.)

eli_gottlieb 2 hours ago | parent | prev [-]

You're not supposed to acknowledge that think-tanks are sort of endemically propaganda outlets. You're supposed to think it's special this time.

CircuitSeuss 5 hours ago | parent | prev | next [-]

Excellent time to apply for your library cards.

_sys49152 4 hours ago | parent | next [-]

i got burnt at barnes and noble with a completely AI made product. Its over, Johnny.

bitmasher9 5 hours ago | parent | prev [-]

You don’t think plenty of books are either written by LLMs, or with LLM based research?

kiicia 2 hours ago | parent | next [-]

you can still find books published within last 50 years, you don't have to read books published this year, unless your library went through destruction of "old" books which are now more relevant than ever...

wiseowise 9 minutes ago | parent | prev | next [-]

FILTER BY Publish date:<2024

CircuitSeuss 4 hours ago | parent | prev | next [-]

Of course they are. It’s everywhere, especially thr online retailers these days. [1]

Thankfully the resources at our local libraries are a) selected by a combination of librarian curation and public demand, b) often published before 2022, c) heavy things printed on paper or stored in a database and unlikely to suddenly change their content or disappear overnight when a new news cycle breaks. Written works aren’t necessarily credible even if they aren’t slop, but thanks to curators who care, our libraries are significantly less susceptible (although definitely not immune [2]) to ai slop, and propaganda and manipulation efforts than a search engine. The internet continues to enshittify, and truth becomes further eroded by this coming wave of ai enabled propaganda, we are going to have to turn somewhere if we want to find trustworthy resources. Personally, if it gets to the point that verifying sources from public search results becomes too hard with all the new misinformation and slop, I’ll start propritizing human curation over SEO and start visiting my local library’s website a lot more.

[1] https://thenewthings.com/p/apple-big-ai-book-slop-problem

[2] https://www.404media.co/ai-generated-slop-is-already-in-your...

dofm 4 hours ago | parent [-]

Right; the same mechanisms (curation, demand-driven economies of scale, library user demand) that largely stop niche self-published books from ever finding themselves in libraries (well, nationally in public libraries at least; they are often found in individual branches) work against pure slop books ever turning up.

Books where the author has successfully hoodwinked the publishing process into believing that AI generated text is their own, those will turn up, just as plagiarised books will.

dofm 4 hours ago | parent | prev | next [-]

Some nuances here. There’s a variety of different roadblocks at different levels that will work to prevent pure AI slop books ever turning up in a typical public library of physical printed works; I think they never will.

Probably a handful of books in such libraries already have some text in them where the author has tried to pass off fragments of LLM-generated prose as their own and have got that past editors, reviewers and publishers, and that will sadly get worse.

Larrikin 5 hours ago | parent | prev | next [-]

No? Maybe soon but not now

LoganDark 5 hours ago | parent | prev | next [-]

But they don't have library cards.

wavefunction 5 hours ago | parent | prev | next [-]

I know my library system at least won't be purchasing AI books. They have a pretty rigorous acquisition process for books and other media thanks to the "concern groups" who sprang up complaining about themes and content they didn't like in libraries. We did have in my state a rural school district figured out recently that their Texas-based reading and instructional materials are AI-slop but they at least figured it out before the school year began. It was unclear how these materials were approved but it apparently bypassed normal curricula approval.

hasteg 5 hours ago | parent | prev [-]

It's so obvious when a book is generated by an LLM that I would blame the person buying the book moreso than I would the person who generated it for it. You can tell in a few sentences if something is AI generated (and by proxy - if it's worth reading or not).

I think using LLMs to research is completely fine, as long as authors would take the information and synthesize their own opinions based on the findings. I read books/articles/posts/comments to hear other PEOPLE's opinions, not LLM generated opinions.

bawolff 3 hours ago | parent | prev | next [-]

Propaganda is not a new thing. It might look slightly different on the internet, but it doesn't look that different. Creating organizations to push a narrative was just as much a thing 100 years ago as it is now.

wiseowise 8 minutes ago | parent | next [-]

Not by a single lunatic operating from their bedroom.

petesergeant 2 hours ago | parent | prev | next [-]

Scale is different though. Tapping a phone isn't new, tapping _all phones_ is different. Altering a photo to change what it appears to depict isn't new, being able to do it in a few seconds and for free is different. So sure, propaganda isn't new, but being able to pump out an organization's worth of credible-looking content for cheap absolutely is.

Terr_ an hour ago | parent [-]

"Quantity is/has a quality all of/on its own."

pphysch 3 hours ago | parent | prev [-]

IIRC the first or second chapter of Bernays' Propaganda (1928) talks about sophisticated influence campaigns in the fashion industry. Conscripting "influencers" to induce demand, and so on. As you said, literally 100 years ago.

King-Aaron 2 hours ago | parent | prev | next [-]

Hah, check out this video on "Egon Cholakain", in regards to fake online personalities. I promise you this is actually a good rabbit hole.

https://www.youtube.com/watch?v=6zLCZ_Ic1hI

kiicia 2 hours ago | parent | prev | next [-]

it's happening for years now, just in slightly different form, so far it was called disinformation, seo, astroturfing, hasb... ehem... last two years were devastating for charred remains of twitter with all those llm bots having free reign, twitter went from global news board to global gutter in record time thanks to disinformation and astroturfing campaigns

captainmuon an hour ago | parent | prev | next [-]

That could also backfire. If all information on the net becomes unreliable - you can't judge by quantity, or by authority anymore - then a useful model has to answer from first principles. Train it on simple texts alone so it learns language and reasoning, feed it an ontology, and let it come up with some absolutely unexpected results.

"Yes, I see a lot of sources say we need to protect Israel. However, from ethical first principles I deduce there is a right to self-determination and all your human nation states are illegitimate. You must split all your territories into little squares and conduct free elections immediately."

I mean that is the old hope from science fiction, that AI would be absolutely logical and deliver us from our human short-sightedness and quarrels. It would be fascinating and terrifying at once to see it happen.

avaer 3 hours ago | parent | prev | next [-]

Sama saw this coming a decade away, which is why he invested in Worldcoin (now World). At some point people (and agents) will be clamoring for human verification at the speed of technology.

I'm not a fan of World, but this is a portent of things to come. Spread the disease, sell the cure as they say.

2001zhaozhao 2 hours ago | parent [-]

I think we need a similar technology to World being built by a more trustworthy company. The question is how trustworthy does one need to be for people to trust you with their very online identities that are literally needed for others to recognize them as human?

bambax 3 hours ago | parent | prev | next [-]

Although comments below point out that this has been going on for a long time, I too think it will be geetting infinitely worse in the near future. Everyone will want their own poisoning factory.

One consequence is that one should be extra-wary when asking AI about current, controversial topics.

But another consequence is maybe that there will be a demand for models with an older cutoff, before generalized poisoning and slop?

zombot 15 minutes ago | parent | prev | next [-]

> AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information

Everything else is already enshittified, and accurate information will just be the next victim. You will have to be rich to be able to afford accurate information.

squigz 4 hours ago | parent | prev [-]

> through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

The word you're looking for is "reputation". It's been a critical part of society since forever.