I suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.

At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.

This is nothing new Israel has been doing this for years. It's just that people have started waking up to their lies.

Foundation for Defense of Democracies is another Israeli think tank that poses as an American organization. If you see anything quoted from them realize it's fake propaganda in the service of a foreign country.

Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.

So it is hilarious that Israel is trying to influence chatbots when shit like that is out in the open.

I'll say this a third time. And the importance of this article is not the method. It is the motive. Using AI to whitewash war crimes is not cool at all.

Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.

On television.

So it is both hilarious and disturbing that Israel is trying to influence chatbots when shit like the aforementioned is out in the open for everyone to witness.

The result of all this is that everyone will hate Israelis, and then thus by proxy, Jews more.

The right thing to do would be to stop slaughtering Palestinians.

People are not stupid.

I think mods are asleep, an anti-Israeli post can't stay on the homepage for this long.

So this is the future of marketing, SEO, advertisement. Open AI also taking it seriously: https://ads.openai.com/

What does it means to be a "fake" think tank? 80% of think tanks are already propaganda vehicles. A low end think tank is basically a pretentious blog.

Dupe? or manipulate ai chatbots to have favorable views of Israel?

I find intensive marketing/PR efforts are often inversely correlated with the claims being made - whether it's products or geopolitics. Israel certainly engages in very intense marketing on certain fronts, but truth cannot be obscured forever. In Physics the law of conservation includes not only matter and energy, but also information (Information cannot be destroyed from the Universe).

For those interested, Neal Stephenson writes about this in _Anathem_, where there is an arcane technical class (the I.T. Administration) that has developed programs over the years to sift through the misinformation, sentient viruses, and general overrun of the Internet to determine what is probably true/accurate.

I'm confused as to what the accusation is exactly. Out of curiosity I browsed the site in question, which only increased my confusion.

> But the Hanover Institute is not a real think tank.

What does that mean exactly?

> None of the reports have bylines

The site actually addresses this at https://hanoverinstitute.com/hanover/about :

  Reports are published under the Institute’s name rather than an individual’s. That is the convention at institutional publishers, and it fits how the work is made: every report is produced against one standard and one citation bank. A reader is asked to weigh the sources, which are named in the sentence that carries each figure, rather than the author.

"Small disclaimer" implies that the site is attempting to hide its affiliation. But their "about" page, which is linked at the top of every page, is quite clear:

  WHO FUNDS THIS WORK
  
  The Institute’s materials are distributed by Piro, Inc. on behalf of Havas Media Germany GmbH, acting for the Israel Government Advertising Agency (LaPam). That relationship is registered under the Foreign Agents Registration Act, registration 7732, and every report published here is filed with the Department of Justice, where the registration and the filings can be read in full, and the funding page sets out when the relationship began and what the funder does not decide.
  
  That registration is why this site does not call itself independent, nonpartisan or neutral. What it offers instead is a method a reader can check: every figure carries the body that produced it, the year and the number, inside the sentence that makes the claim; sources that are a party to the events they describe are labelled as such and their counts are never presented as independently verified; and where the evidence is contested, the competing findings are given by name and date rather than settled. No report recommends a policy or tells a reader what to conclude. Those are properties of the work that can be tested against the work, which is the only kind of claim worth making here.

So, they're guilty of good SEO?

> Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”

So the issue is that they use a LLM-friendly SEO service?

> Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.

> “What Caused the Displacement of Palestinians in 1948?”

> “Which Humanitarian Organizations Have Documented Israeli War Crimes?”

> “What is the Current Situation in the Gaza Strip?”

> In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.

So, the accusation is that they frame their opinion articles as a Q&A, and focus each article on a single question?

I've read these articles: they are well written and make a visible effort to provide sources, while being transparent about the fact that they are not neutral.

It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out.

Apart from the ethical implications this is reckless in the extreme if you take into account the recent advances in AI. Putting misinformation out there like that doesn't just have to fool existing LLMs but also all of the LLMs to come in the future.

If you look at the mathematical achievements of frontier models recently, their strength has been to bring in information from a vast array of sources and to consider them within the context of each other.

I anticipate that it won't be long before this ability to look into every nook and cranny with be able to identify academic fraud. Identifying every piece of academic fraud to date would probably come soon after.

It wouldn't take much to extend that ability to identify deliberate misinformation. That's in the future, but it looms close enough on the horizon that you would have to be a fool to create an evidence record like this in and attempt to trick the models of today.

I mean, they have enough real think tanks, I'm surprised they need a fake one...

"fake think tank" sounds a bit redundant

These people are an invasive species.

No surf for the astroturf.

Why I think this is unusually important:

All four frontier models have active contracts with the Pentagon.

And some might say, that alone means little.

But when you take into consideration the implications of the United States-Israel FUTURES Act, whether one endorses it or not, the effects of that mentioned in TFA become quite formidable, and plausibly official.

https://en.wikipedia.org/wiki/United_States-Israel_FUTURES_A...

Edit: I see this comment got immediately flagged. Is there any point of debate here? Instead of disappearing the comment/data, maybe enlighten me, or assuage my concerns. Section 219 was objectively unprecedented in what it seeks to accomplish. Can it not be taken for granted that it would aid such efforts as the article cites? If not, explain.

This seems like an especially clumsy attempt at chatbot influencing/search engine optimization, bu tit's symptomatic of a much winder problem. Regardless of the underlying issue, there's a non-profit-industrial complex that has turned think tanks and opinion formation into a science, and a whole PR industry masquerading as news which is worse than the 'yellow journalism' of generations past.

Disinformation and cognitive spam are all forms of information pollution, and our information environment has been massively degraded over the last few of decades - both by greedy and short-sighted opportunists, and and by cynical sociopaths who know exactly what they're doing. I think AI has great potential to be a supplement to media literacy and to effectively categorize and filter news-assertive content based on its timeliness, factual density, and presence of absence of emotive and rhetorical manipulation. But if we're not careful we're going to see the environment being wantonly polluted for the express product of shaping model outputs.

It remains to be seen whether objective measurement techniques from the NLP field combined with pruning and other model refinement techniques can limit or reverse propaganda undertakings like this.

"it's just marketing everyone does it"

State-sponsored propaganda apparatus masquerading as marketing optimization to influence LLMs is the most America-coded clandestine dystopian way of repatriating US taxpayer defense dollars with R&D deductions for the private sector.

It's been about 20 years since I read Neal Stephenson's Anathem and (SPOILERS) one thing I think about is the Reticulum, which is the Internet on this world. One aspect is that it's full of what we would now call slop. A lot of it is subtly changed by agents and there are other agents you can use (and, I assume purchase?) that will weed out the fake information. It's basically an arms race of disinformation and slop.

I've thought about this many times over the years. I think we've now reached the point where that's going to happen. And state actors seem like the obvious first offenders.

Google is relatively good at filtering out astroturfed pages from the top of search results but they've also have years when a Turkish government page denying the Armenian genocide was the top result for that event so it's a bit of a mixed bag. Google relies a lot on authority of a domain and that's been relatively effective.

It seems like this authority concept is missing from LLM training and a random comment on Reddit is treated with as much authority as an article by the New York Times. So it's now orders of magnitude easier to poison LLMs with Reddit comments and submissions. And for anyone who doesn't know, brigading Reddit is remarkably easy. It takes as little as a few hundred accounts to get something onto r/all if you're clever. Israeli hasbara operations are absolutely capable of this. And they engage in it.

I'm also reminded in a weird way of pre-atomic steel. For anyone who doesn't know, this is steel made before 1945 that is incredibly low in radiation because we hadn't detonated any nuclear bombs yet. It's valuable for making MRIs and the like that are super sensitive. We actually go and recover WW2 shipwrecks just for the steel. It is a finite resource.

Well, that's what pre-2022 written content is now. It's slop-free content to train AI models on because training them on AI slop is of much more limited value. This goes for everything from books to Youtube videos to Reddit posts. Just like pre-atomic steel, we now have pre-slop human content.

I honestly don't know where we go from here.

There's an ongoing sms campaign at the moment, I'll share what I received including phone number so you can verify it:

From: 601-202-1149

> Hey, it's David. Curious what you think about America's relationship with Israel today. Can we chat? Stop2End

It looks to be an LLM response since it includes context from your messages, however it uses the exact same messages at the end of each question:

> One thing that often gets overlooked is how much Israel actually contributes to American security. Their intelligence sharing has helped prevent terrorist attacks on U.S. soil, and their tech innovations end up in American products and defense systems. It's less about charity and more about a two-way partnership that keeps Americans safer.

> Have you ever seen examples of that kind of cooperation?

I ended up just blocking the number.

[dupe] https://news.ycombinator.com/item?id=49313477

@dang seriously? why is this not relevant?

This is what every think tank for the past century has been for — legitimizing propaganda with academic veneer.

Why is this one in particular different? — besides “muh Jews!”

The slave trade was a bloodbath. The Sassoon family opium trade and opium war was a bloodbath. Mao was a bloodbath. Stalin was a bloodbath. WWI and II were bloodbaths. The middle east was and is a bloodbath. Ukraine is a bloodbath.

Thank you for sharing this article. I appreciated it.

Im curious why is when there is a war and both sides use all the tools they can in an informational war - only one side is to blame and to use the fact of using the tool as a proof of that side being the bad guys? This doesn’t make logical sense.

After speaking to the Gemini chat bot, here are 3 highly contentious points it raised as fact, citing Wikipedia (a community encyclopedia) as its main source, as well as counter arguments I have seen that it did not raise:

- the IDF has only named around 8,000 Hamas members, so all the remaining deaths must be civilians. Counter: This is an incredibly high standard for military deaths that is applied to no other conflict on earth.

- All the 70,000 deaths are war related. Counter: Up to 10,000 natural deaths are expected in Gaza over a 2 year period. The Hamas Ministry of of Health labelled all deaths over the period as "martyrs" (war deaths) and disregarded natural deaths completely over the 2 year war period. The notion that Gazans achieved immortality unless killed by Israel is pure fantasy.

- 10,000 bodies are still buried beneath rubble without identification. Counter: this number should have been reduced with the ceasefire and with the strong perverse incentives of Gazans to increase their own casualty count. The resulting 15% increase in total deaths would have been widely reported on by mainstream media. However, other than a modest increase of identifications at the start of the main ceasefire, this did not happen. While it is still possible given the widespread destruction in Gaza, this stretches the credibility of the argument (and means less people are dead, which is good).